-- Logs begin at Mon 2022-05-23 11:48:20 UTC, end at Mon 2022-05-23 12:54:35 UTC. -- May 23 12:17:46.622762 dv-cinder-836996-27-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 23 12:17:46.631133 dv-cinder-836996-27-iscsi devstack@keystone.service[80203]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 23 12:17:46.633221 dv-cinder-836996-27-iscsi devstack@keystone.service[80203]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon May 23 12:17:46 2022] *** May 23 12:17:46.633221 dv-cinder-836996-27-iscsi devstack@keystone.service[80203]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 23 12:17:46.633221 dv-cinder-836996-27-iscsi devstack@keystone.service[80203]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 23 12:17:46.633221 dv-cinder-836996-27-iscsi devstack@keystone.service[80203]: nodename: dv-cinder-836996-27-iscsi May 23 12:17:46.633221 dv-cinder-836996-27-iscsi devstack@keystone.service[80203]: machine: x86_64 May 23 12:17:46.633221 dv-cinder-836996-27-iscsi devstack@keystone.service[80203]: clock source: unix May 23 12:17:46.633221 dv-cinder-836996-27-iscsi devstack@keystone.service[80203]: pcre jit disabled May 23 12:17:46.633221 dv-cinder-836996-27-iscsi devstack@keystone.service[80203]: detected number of CPU cores: 4 May 23 12:17:46.633221 dv-cinder-836996-27-iscsi devstack@keystone.service[80203]: current working directory: / May 23 12:17:46.633221 dv-cinder-836996-27-iscsi devstack@keystone.service[80203]: detected binary path: /usr/bin/uwsgi-core May 23 12:17:46.633221 dv-cinder-836996-27-iscsi devstack@keystone.service[80203]: your processes number limit is 47680 May 23 12:17:46.633221 dv-cinder-836996-27-iscsi devstack@keystone.service[80203]: your memory page size is 4096 bytes May 23 12:17:46.633221 dv-cinder-836996-27-iscsi devstack@keystone.service[80203]: detected max file descriptor number: 2048 May 23 12:17:46.633221 dv-cinder-836996-27-iscsi devstack@keystone.service[80203]: lock engine: pthread robust mutexes May 23 12:17:46.633955 dv-cinder-836996-27-iscsi devstack@keystone.service[80203]: thunder lock: enabled May 23 12:17:46.633955 dv-cinder-836996-27-iscsi devstack@keystone.service[80203]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 23 12:17:46.633955 dv-cinder-836996-27-iscsi devstack@keystone.service[80203]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 23 12:17:46.678341 dv-cinder-836996-27-iscsi devstack@keystone.service[80203]: Python main interpreter initialized at 0x5601480769c0 May 23 12:17:46.678341 dv-cinder-836996-27-iscsi devstack@keystone.service[80203]: python threads support enabled May 23 12:17:46.678341 dv-cinder-836996-27-iscsi devstack@keystone.service[80203]: your server socket listen backlog is limited to 100 connections May 23 12:17:46.678341 dv-cinder-836996-27-iscsi devstack@keystone.service[80203]: your mercy for graceful operations on workers is 90 seconds May 23 12:17:46.678705 dv-cinder-836996-27-iscsi devstack@keystone.service[80203]: mapped 403077 bytes (393 KB) for 2 cores May 23 12:17:46.678781 dv-cinder-836996-27-iscsi devstack@keystone.service[80203]: *** Operational MODE: preforking *** May 23 12:17:46.679033 dv-cinder-836996-27-iscsi devstack@keystone.service[80203]: *** uWSGI is running in multiple interpreter mode *** May 23 12:17:46.679033 dv-cinder-836996-27-iscsi devstack@keystone.service[80203]: spawned uWSGI master process (pid: 80203) May 23 12:17:46.679238 dv-cinder-836996-27-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 23 12:17:46.682579 dv-cinder-836996-27-iscsi devstack@keystone.service[80203]: spawned uWSGI worker 1 (pid: 80215, cores: 1) May 23 12:17:46.682579 dv-cinder-836996-27-iscsi devstack@keystone.service[80203]: spawned uWSGI worker 2 (pid: 80216, cores: 1) May 23 12:17:46.682579 dv-cinder-836996-27-iscsi devstack@keystone.service[80203]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 23 12:17:52.321663 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 12:17:52.321663 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: from cryptography.utils import int_from_bytes May 23 12:17:52.322415 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 12:17:52.322415 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: from cryptography.utils import int_from_bytes May 23 12:17:52.361840 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 12:17:52.361840 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: from cryptography.utils import int_from_bytes May 23 12:17:52.362394 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 12:17:52.362394 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: from cryptography.utils import int_from_bytes May 23 12:17:52.372028 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 23 12:17:52.372247 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 23 12:17:52.372446 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] command line args: [] {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 23 12:17:52.372719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 23 12:17:52.372901 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ================================================================================ {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 23 12:17:52.373157 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] admin_token = **** {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.373408 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] config_dir = [] {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.373599 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.373786 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] config_source = [] {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.373969 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] debug = True {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.374170 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.374662 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.374745 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.374966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.375216 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.375448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] list_limit = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.375629 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] log_config_append = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.375809 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.376002 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] log_dir = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.376164 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] log_file = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.376424 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.376672 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.376922 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.377169 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.377350 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.377529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.377774 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.378020 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.378254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.378488 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.378721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.378955 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.379202 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.379448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.379727 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.380244 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.380433 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] publish_errors = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.380669 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.380903 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.381085 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.381337 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.381571 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.381808 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] standard_threads = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.382044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.382291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.382470 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.382649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] use_journal = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.382828 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] use_json = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.383006 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] use_stderr = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.383209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] use_syslog = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.383446 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.383713 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.383949 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.384232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.384481 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.384733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.384972 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.385221 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.385460 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.385710 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.385959 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.386213 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.386451 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.386700 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.386941 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.387189 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.387438 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.387677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.387933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.388216 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.388470 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.388711 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.388967 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.389230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.389486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.389737 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.389969 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.390201 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.390433 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.390672 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.390925 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.391178 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.391414 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.391647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.391895 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.392144 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.392409 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.392660 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.392896 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.393134 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.393371 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.393605 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.393838 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.394070 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.394303 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.394541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.394792 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.395023 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.395260 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.395493 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.395731 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.395971 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.396246 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.396483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.396720 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.396959 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.397196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.397433 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.397668 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.397901 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.398154 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.398393 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.398632 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.399004 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.399264 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.399506 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.399741 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.399980 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.400241 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.400515 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.400756 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.401007 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.401262 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.401503 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.401743 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.401983 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.402233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.402473 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.402711 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.402959 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.403200 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.403440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.403680 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.403917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.404155 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.404439 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.404683 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.404929 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.405168 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.405414 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.405653 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.405890 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.406127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.406360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.406593 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.406826 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.407065 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.407299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.407542 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.407780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.408030 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.408292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.408551 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.408811 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.409055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.409288 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] auth.external = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.409523 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.409820 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.410055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.410289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] auth.password = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.410522 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] auth.token = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.410763 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.411001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.411257 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.411341 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 23 12:17:52.411492 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.411569 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 23 12:17:52.411698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] command line args: [] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 23 12:17:52.411775 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.411971 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 23 12:17:52.412047 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.412151 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ================================================================================ {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 23 12:17:52.412256 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.412432 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] admin_token = **** {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.412508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] credential.caching = True {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.412681 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] config_dir = [] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.412757 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.412869 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.412952 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.413052 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] config_source = [] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.413166 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.413241 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] debug = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.413405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.413487 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.413651 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.413730 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.413829 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.413904 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.414080 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.414171 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.414326 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.414906 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.415082 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] list_limit = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.415191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.415286 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] log_config_append = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.415386 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.415591 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.415692 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.415786 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] log_dir = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.415878 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.415970 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] log_file = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.416139 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.416272 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.416412 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.416491 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.416666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.416781 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.416952 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] federation.caching = True {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.417066 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.417066 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.417248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.417369 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.417453 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.417525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.417645 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.417720 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.417796 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.417911 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.418043 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.418122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.418282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.418368 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.418538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.418626 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.418777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.418863 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.419030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.419107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.419265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.419346 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.419514 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.419753 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] identity.caching = True {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.419830 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.419963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] publish_errors = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.420038 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.420214 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.420299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.420449 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.420535 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.420628 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.420705 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.420874 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.420957 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.421110 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.421219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.421346 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] standard_threads = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.421451 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.421580 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.421664 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.421822 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.421906 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.422000 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.422084 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.422165 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] use_journal = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.422316 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.422396 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] use_json = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.422517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] use_stderr = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.422593 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.422682 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] use_syslog = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.422827 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.422910 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 12:17:52.423020 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.423158 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.423268 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.423400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.423550 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.423660 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.423772 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.423885 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.424045 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.424122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.424310 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.424389 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.424550 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.424633 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.424784 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.424861 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.425023 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.425100 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.425255 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.425331 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.425495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.425570 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.425738 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.425824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.425977 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.426055 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.426241 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.426325 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.426457 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.426540 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.426704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.426780 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.426937 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.427013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.427175 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.427258 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.427422 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.427499 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.427668 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.427752 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.427903 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.427986 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.428156 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.428274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.428438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.428515 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.428693 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.428781 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.428942 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.429020 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.429174 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.429251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.429407 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.429496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.429639 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.429733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.429877 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.429955 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.430877 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.431024 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.431156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.431293 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.431373 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.431564 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.431642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.431801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.431879 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.432043 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.432121 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.432315 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.432403 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.432553 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.432630 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.432786 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.user = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.432870 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.433025 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.433101 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.433278 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.433355 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.433510 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.433589 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.433741 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.433818 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.433974 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.434051 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.434206 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.434285 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.434437 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.434514 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.434672 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.434748 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.434911 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.434990 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.435140 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.435217 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.435372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.435448 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.435601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.435679 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.435837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.435914 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.436075 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.436152 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.436345 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.436425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.436576 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.436661 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.436813 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.436898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.437043 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.437119 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.437277 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.437352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.437512 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.437599 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.437747 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.437824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.437976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.438054 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.438209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.438358 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.438433 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.438504 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.438674 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.438752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.438904 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.438991 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.439158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.439236 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.439390 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.439467 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.439618 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.439694 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.439855 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.439934 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.440090 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.440191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.440353 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.440430 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.440605 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.440683 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.441092 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.441176 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.441336 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.441422 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.441571 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.441653 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.441808 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.441889 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] resource.caching = True {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.442046 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.442135 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.442294 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.442372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.442529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.442605 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.442762 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.442845 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.443019 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.443095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.443254 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.443329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.443490 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.443576 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.443727 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.443813 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.443960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.444036 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.444215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] role.caching = True {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.444310 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.444465 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] role.driver = sql {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.444543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.444697 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.444774 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.444937 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.445022 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.445176 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.445253 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.445419 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.445496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.445687 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.445811 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.445937 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.446018 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.446260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.446388 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.446511 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.446660 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.446788 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.446914 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.447040 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.447166 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.447298 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.447433 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.447559 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.447684 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.447809 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.447935 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.448061 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.448216 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.448351 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.448351 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.448581 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.448702 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.448823 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.448953 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.449076 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.449197 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.449319 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.449440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] auth.external = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.449561 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.449710 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.449833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.449955 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.450088 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.450210 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.450340 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.450463 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] auth.password = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.450584 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.450705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] auth.token = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.450825 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.450946 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.451075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.451205 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.451326 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.451448 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.451569 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.451699 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.451829 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.451951 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.452073 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.452217 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.452340 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.452462 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.452594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.452716 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] credential.caching = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.452845 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.452967 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.453090 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.453210 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.453344 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] token.caching = True {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.453476 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.453599 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.453720 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.453841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.453963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.454093 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.454221 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.454342 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.454469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.454592 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.454712 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.454841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.454964 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.455096 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.455218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.455339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.455461 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.455591 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.455713 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.455835 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.455956 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.456078 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.456276 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.456407 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.456534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.456660 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.456785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] federation.caching = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.456919 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.457044 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.457182 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.457313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.457439 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.457563 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.457688 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.457814 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.457946 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] database.connection = **** {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.458080 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.458214 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.458340 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.458464 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.458589 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.458728 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.458866 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.458991 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.459115 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.459241 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.459365 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.459489 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.459685 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.459811 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.459935 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity.caching = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.460059 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.460201 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.460324 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.460464 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.460587 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.460707 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.460828 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.460948 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.461069 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.461196 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.461348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.461519 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.461642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.461768 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.461894 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.462027 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.462150 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.462281 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.462403 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.462524 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.462644 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.462773 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.462895 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80216) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 23 12:17:52.463024 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.463024 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.463024 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.463024 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.463024 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.463024 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.463024 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.463801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.463801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.463801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.464141 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.464141 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.464437 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.464628 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.464868 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.465117 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.465348 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.465595 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.465843 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.466081 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.466329 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.466575 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.466806 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.467044 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.467275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.467509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.467750 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.467992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.468250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.468489 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.468739 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.468986 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.469218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.469448 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.469695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.469964 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.470205 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.470445 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.470684 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.470917 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.471160 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.471419 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.471652 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.471899 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.472148 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.472418 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.472656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.472886 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.473123 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.473359 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.473594 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.473926 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.474188 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.474442 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.474693 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.474941 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.475188 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.475421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.475666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.475905 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.476143 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.476408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.476646 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.477448 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.477780 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.478023 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.478282 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.478516 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.478764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.479004 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.479241 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.479481 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.479729 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.479973 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.480231 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.480472 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.480714 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] resource.caching = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.480966 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.481214 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.481450 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.481698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.481950 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.482186 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.482432 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.482670 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.482910 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.483146 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] role.caching = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.483394 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] role.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.483624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.483873 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.484121 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.484394 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.484641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.484887 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.485133 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.485379 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.485627 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.485860 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.486107 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.486354 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.486599 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.486846 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.487255 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.487494 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.487750 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.488000 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.488276 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.488530 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.488767 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.489020 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.489271 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.489510 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.489743 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.489977 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.490210 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.490460 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.490715 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.490964 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.491201 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.491439 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.491669 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.491906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] token.caching = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.492143 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.492413 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.493191 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.493447 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.493663 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80216) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.493776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.493899 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80216) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.494042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.494120 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80216) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.494290 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.494536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.494786 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.495023 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.495261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.495499 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.495746 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.495995 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.496252 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.496500 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.496795 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.497070 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.connection = **** {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.497318 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.497565 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.497808 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.498052 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.498290 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.498526 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.498760 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.498995 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.499240 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.499477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.499720 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.499956 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.500223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.500464 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.500707 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.500937 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.501179 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.501416 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 12:17:52.501586 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80215) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 23 12:17:52.502801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80216) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.527227 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.527457 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.527685 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.532514 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80216) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.532753 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80216) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.537652 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.557471 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.557698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.586479 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80216) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.595116 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.638193 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80216) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.638426 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80216) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.639769 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.640006 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.704556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80216) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.704790 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80216) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.705005 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.705109 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80216) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.705229 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80216) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.705309 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.705448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80216) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.705529 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.705664 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.705755 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80216) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.705882 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.706096 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.728830 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80216) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.729065 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80216) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.729286 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80216) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.729370 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.729505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80216) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.729586 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.729719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80216) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.729810 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.729932 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80216) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.730014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.730145 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80216) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.730225 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.730402 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.730616 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.741505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80216) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.741801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.754403 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80216) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.754642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80216) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.754852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.754948 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80216) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.755083 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80216) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.755184 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.755299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80216) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.755393 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.755513 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80216) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.755595 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.755727 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.755805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80216) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.755941 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.756155 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80215) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 12:17:52.773629 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.774571 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 12:17:52.774925 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.775087 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.775202 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 12:17:52.775286 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 12:17:52.775286 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.775420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.775492 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 12:17:52.775572 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.775707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.775789 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.775956 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.776036 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.776234 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.776491 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.787448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.787725 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.787996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.788305 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.788717 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.788992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.789262 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.789707 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.796601 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80216) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.797800 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.799856 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80216) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.800134 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.800445 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.800728 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.801009 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.801101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.801270 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.801350 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.801539 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.801619 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.801876 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.801956 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.802157 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.802235 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.802414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.802493 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.802693 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.802793 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.803068 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.803339 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.803609 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.803875 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.829871 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.831202 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.831582 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80216) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.831862 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.832942 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.833224 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.837801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80216) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.838034 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.838317 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.838978 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.839256 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.839535 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.845581 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80216) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.845868 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.846860 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.847141 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.850324 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80216) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.850723 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.851004 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.851288 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.851565 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.851648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.851862 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.852142 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.852320 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.852462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.852619 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.852714 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.853028 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.853301 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.853566 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.853832 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.871801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.872072 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.876052 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.876357 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.876766 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80216) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.880379 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.880676 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.881027 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.885003 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.885295 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.885798 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80216) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.889416 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80216) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.890407 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.893682 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.893980 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.894270 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.894560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.894639 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.894845 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.895130 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.898426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.898752 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.899056 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.899333 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.899609 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.899893 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.914052 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80216) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.914354 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.914809 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.915064 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.919052 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.919335 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.919600 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.919860 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.923054 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.924794 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.925071 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.927851 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.928120 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80216) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.928433 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.928713 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.928990 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.929271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.929571 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.929862 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.933497 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.933775 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.934053 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.934331 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.934606 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.943606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80216) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.943886 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.944187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.944472 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.944753 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.945028 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.945312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.948732 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.949011 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.949289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.949571 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.949850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.950130 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.950424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.963987 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80216) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.967269 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80216) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.968505 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.970599 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.971739 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.972228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.973851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80216) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.974138 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.974420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.975051 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.976699 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.978361 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.978641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.978929 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.981822 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.983438 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80216) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.986321 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.986699 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.987001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.987287 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.987565 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.987905 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.991147 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.991434 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.991712 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.992025 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.997361 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80216) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:52.997645 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:52.998028 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:53.001641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:53.001924 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:53.002302 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:53.005852 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80216) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:53.006136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:53.006417 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:53.006691 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:53.006963 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:53.007239 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:53.007534 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:53.007802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:53.008077 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:53.008913 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:53.009188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:53.009442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:53.009719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:53.009986 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80216) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:53.010707 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80215) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 12:17:53.010988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:53.011264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:53.011536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:53.011802 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:53.012075 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:53.012374 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:53.012913 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:53.013192 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:53.013469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:53.013736 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:53.013995 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:53.014253 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:53.014515 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80215) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 12:17:53.048541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5601480769c0 pid: 80216 (default app) May 23 12:17:53.051826 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b7b00-03c1-40aa-a987-0ee13afd0824 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:17:53.052121 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b7b00-03c1-40aa-a987-0ee13afd0824 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:17:53.052428 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b7b00-03c1-40aa-a987-0ee13afd0824 None None] PATH_INFO: `/v3` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:17:53.053137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5601480769c0 pid: 80215 (default app) May 23 12:17:53.057266 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1/1] 10.222.0.16 () {50 vars in 791 bytes} [Mon May 23 12:17:53 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:01.027497 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b753a-3693-4161-8acf-080d78496951 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:01.027901 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b753a-3693-4161-8acf-080d78496951 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:01.028058 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b753a-3693-4161-8acf-080d78496951 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:01.032463 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1/2] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:01.037949 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf258cc-f75d-43f9-a3a0-8906b955e3a5 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:01.038371 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf258cc-f75d-43f9-a3a0-8906b955e3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:01.038504 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf258cc-f75d-43f9-a3a0-8906b955e3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:01.107580 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: INFO dbcounter [None req-aaf258cc-f75d-43f9-a3a0-8906b955e3a5 None None] Registered counter for database keystone May 23 12:18:01.128448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG oslo_db.sqlalchemy.engines [None req-aaf258cc-f75d-43f9-a3a0-8906b955e3a5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80216) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 12:18:01.134918 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writer thread running {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 12:18:01.311157 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG passlib.handlers.bcrypt [None req-aaf258cc-f75d-43f9-a3a0-8906b955e3a5 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80216) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 23 12:18:01.311501 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG passlib.handlers.bcrypt [None req-aaf258cc-f75d-43f9-a3a0-8906b955e3a5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80216) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 23 12:18:01.367456 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-aaf258cc-f75d-43f9-a3a0-8906b955e3a5 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:01.369109 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-aaf258cc-f75d-43f9-a3a0-8906b955e3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:01.492324 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 2/3] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:01 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 23 12:18:01.679099 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b753a-3693-4161-8acf-080d78496951 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:01.747599 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: INFO dbcounter [None req-642b753a-3693-4161-8acf-080d78496951 None None] Registered counter for database keystone May 23 12:18:01.767023 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG oslo_db.sqlalchemy.engines [None req-642b753a-3693-4161-8acf-080d78496951 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80215) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 12:18:01.773123 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writer thread running {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 12:18:01.892828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b753a-3693-4161-8acf-080d78496951 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:01.897020 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b2423-52cc-42d7-a684-be734079d8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:01.897678 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b2423-52cc-42d7-a684-be734079d8f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:01.897962 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b2423-52cc-42d7-a684-be734079d8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:01.898241 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b2423-52cc-42d7-a684-be734079d8f3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:01.906907 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b2423-52cc-42d7-a684-be734079d8f3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:01.959468 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c49b2423-52cc-42d7-a684-be734079d8f3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 23 12:18:01.959746 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b2423-52cc-42d7-a684-be734079d8f3 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:01.969939 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 2/4] 10.222.0.16 () {58 vars in 1324 bytes} [Mon May 23 12:18:01 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 12:18:01.976047 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf258cc-f75d-43f9-a3a0-8906b955e3a5 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:01.990868 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf258cc-f75d-43f9-a3a0-8906b955e3a5 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:01.995476 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d05daa-58c1-4029-82e7-4d7290ea8d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:01.996630 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d05daa-58c1-4029-82e7-4d7290ea8d91 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:01.996917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d05daa-58c1-4029-82e7-4d7290ea8d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:01.997202 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d05daa-58c1-4029-82e7-4d7290ea8d91 admin admin] PATH_INFO: `/v3/projects/c631b5a3911e4647a05ad6138d2bc618` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:01.998671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d05daa-58c1-4029-82e7-4d7290ea8d91 admin admin] RBAC: Authorizing `identity:get_project(project_id=c631b5a3911e4647a05ad6138d2bc618)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:02.159381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: WARNING keystone.common.rbac_enforcer.enforcer [None req-80d05daa-58c1-4029-82e7-4d7290ea8d91 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 23 12:18:02.160053 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d05daa-58c1-4029-82e7-4d7290ea8d91 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:02.164466 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 3/5] 10.222.0.16 () {56 vars in 1207 bytes} [Mon May 23 12:18:01 2022] GET /identity/v3/projects/c631b5a3911e4647a05ad6138d2bc618 => generated 344 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:04.346165 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a9df3-6734-48b0-8a7a-3ad842b71ea7 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:04.346665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a9df3-6734-48b0-8a7a-3ad842b71ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:04.346665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a9df3-6734-48b0-8a7a-3ad842b71ea7 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:04.350725 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 3/6] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:18:04.356710 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba547793-8324-4620-9e88-7423c35bade0 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:04.357106 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba547793-8324-4620-9e88-7423c35bade0 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:04.357278 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba547793-8324-4620-9e88-7423c35bade0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:04.376761 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-ba547793-8324-4620-9e88-7423c35bade0 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:04.378281 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-ba547793-8324-4620-9e88-7423c35bade0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:04.388589 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 4/7] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:04 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 23 12:18:04.422026 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a9df3-6734-48b0-8a7a-3ad842b71ea7 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:04.434970 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a9df3-6734-48b0-8a7a-3ad842b71ea7 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:04.439103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae9b8b-75bb-4edd-a817-a91933d8b671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:04.439736 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae9b8b-75bb-4edd-a817-a91933d8b671 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:04.440022 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae9b8b-75bb-4edd-a817-a91933d8b671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:04.440342 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae9b8b-75bb-4edd-a817-a91933d8b671 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:04.448399 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae9b8b-75bb-4edd-a817-a91933d8b671 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:04.450168 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae9b8b-75bb-4edd-a817-a91933d8b671 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:04.458796 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 4/8] 10.222.0.16 () {58 vars in 1324 bytes} [Mon May 23 12:18:04 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 12:18:04.464545 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba547793-8324-4620-9e88-7423c35bade0 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:04.473645 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba547793-8324-4620-9e88-7423c35bade0 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:04.478832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbafea9a-4a80-463b-991d-86d5f0adb3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:04.479764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafea9a-4a80-463b-991d-86d5f0adb3d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:04.480056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafea9a-4a80-463b-991d-86d5f0adb3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:04.480373 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafea9a-4a80-463b-991d-86d5f0adb3d5 admin admin] PATH_INFO: `/v3/users/89b57772ff384ce6af49426c38787139` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:04.481781 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbafea9a-4a80-463b-991d-86d5f0adb3d5 admin admin] RBAC: Authorizing `identity:get_user(user_id=89b57772ff384ce6af49426c38787139)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:04.483632 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbafea9a-4a80-463b-991d-86d5f0adb3d5 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:04.485303 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 5/9] 10.222.0.16 () {56 vars in 1198 bytes} [Mon May 23 12:18:04 2022] GET /identity/v3/users/89b57772ff384ce6af49426c38787139 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:06.728480 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cee662-0374-4c95-b6a9-7ef3a2e46f75 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:06.728995 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cee662-0374-4c95-b6a9-7ef3a2e46f75 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:06.729095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cee662-0374-4c95-b6a9-7ef3a2e46f75 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:06.730377 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 5/10] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:18:06.735539 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8a1a0-315b-4d4d-8fc2-06edaffa8a0f None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:06.735958 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8a1a0-315b-4d4d-8fc2-06edaffa8a0f None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:06.736109 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf8a1a0-315b-4d4d-8fc2-06edaffa8a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:06.753141 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-dcf8a1a0-315b-4d4d-8fc2-06edaffa8a0f None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:06.754306 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-dcf8a1a0-315b-4d4d-8fc2-06edaffa8a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:06.771299 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e033ba-5acd-4392-abf3-5b0c1fd97a69 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:06.771592 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e033ba-5acd-4392-abf3-5b0c1fd97a69 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:06.771878 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e033ba-5acd-4392-abf3-5b0c1fd97a69 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:06.773253 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 6/11] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:18:06.778354 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5956dd-a9b0-4090-a4c3-291df9dd1597 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:06.778647 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5956dd-a9b0-4090-a4c3-291df9dd1597 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:06.778934 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5956dd-a9b0-4090-a4c3-291df9dd1597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:06.819588 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG passlib.handlers.bcrypt [None req-9a5956dd-a9b0-4090-a4c3-291df9dd1597 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80215) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 23 12:18:06.820031 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG passlib.handlers.bcrypt [None req-9a5956dd-a9b0-4090-a4c3-291df9dd1597 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80215) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 23 12:18:06.834330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 6/12] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:06.857548 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf8a1a0-315b-4d4d-8fc2-06edaffa8a0f None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:06.866790 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-9a5956dd-a9b0-4090-a4c3-291df9dd1597 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:06.869045 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-9a5956dd-a9b0-4090-a4c3-291df9dd1597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:06.880938 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:06.962022 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf8a1a0-315b-4d4d-8fc2-06edaffa8a0f None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:06.970183 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 7/13] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 12:18:06.992634 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5956dd-a9b0-4090-a4c3-291df9dd1597 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:06.995303 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b278fa-f42e-4287-9283-76e3c81e8aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:06.996108 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b278fa-f42e-4287-9283-76e3c81e8aef None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:06.996420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b278fa-f42e-4287-9283-76e3c81e8aef None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:06.996698 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b278fa-f42e-4287-9283-76e3c81e8aef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:07.008056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b278fa-f42e-4287-9283-76e3c81e8aef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:07.019197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-75b278fa-f42e-4287-9283-76e3c81e8aef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:18:07.019197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:07.019197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:07.019197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:07.019197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:07.019197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:07.019197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:07.019197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:07.019197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:07.019197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:07.019197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:07.019197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:07.019197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:07.019197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:07.019197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:07.019197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:07.020233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:07.020233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:07.020233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:07.020233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:07.020233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:07.020233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:07.020233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:07.020233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:07.020233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:07.020233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:07.020233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:07.020233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:18:07.020233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:07.020233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:07.020233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:07.020233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:07.021092 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:07.021092 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:07.021092 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:07.021092 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:07.021092 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:07.021092 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:07.021092 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:07.021092 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:07.021092 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:07.021092 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:07.021092 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:18:07.021092 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:18:07.021092 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 7/14] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:18:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:18:07.026990 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b278fa-f42e-4287-9283-76e3c81e8aef None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:07.097694 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5956dd-a9b0-4090-a4c3-291df9dd1597 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:07.100687 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:07.107831 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:07.127258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b278fa-f42e-4287-9283-76e3c81e8aef None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:07.135821 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a00e3f3-a305-40a5-b423-404015f0dafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:07.136749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00e3f3-a305-40a5-b423-404015f0dafc None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:07.137043 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00e3f3-a305-40a5-b423-404015f0dafc None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:07.137346 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00e3f3-a305-40a5-b423-404015f0dafc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:07.161078 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c5c39-3555-4ee5-9258-b67b6cec8849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:07.161737 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c5c39-3555-4ee5-9258-b67b6cec8849 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:07.162019 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c5c39-3555-4ee5-9258-b67b6cec8849 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:07.162296 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0c5c39-3555-4ee5-9258-b67b6cec8849 None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:07.163051 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c5c39-3555-4ee5-9258-b67b6cec8849 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:07.163358 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c5c39-3555-4ee5-9258-b67b6cec8849 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:07.165694 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0c5c39-3555-4ee5-9258-b67b6cec8849 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:07.177269 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 8/15] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:18:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:18:07.183039 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c5c39-3555-4ee5-9258-b67b6cec8849 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:07.205990 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a00e3f3-a305-40a5-b423-404015f0dafc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:07.207515 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a00e3f3-a305-40a5-b423-404015f0dafc None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:07.281682 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0c5c39-3555-4ee5-9258-b67b6cec8849 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:07.303324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 8/16] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:18:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 12:18:07.309531 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a00e3f3-a305-40a5-b423-404015f0dafc None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:07.312163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:07.315005 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097af87-b312-420a-b1d0-6a191da59ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:07.315884 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097af87-b312-420a-b1d0-6a191da59ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:07.316452 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097af87-b312-420a-b1d0-6a191da59ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:07.316734 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097af87-b312-420a-b1d0-6a191da59ab7 None admin] PATH_INFO: `/v3/users/89b57772ff384ce6af49426c38787139` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:07.317940 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9097af87-b312-420a-b1d0-6a191da59ab7 None admin] RBAC: Authorizing `identity:get_user(user_id=89b57772ff384ce6af49426c38787139)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:07.319408 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9097af87-b312-420a-b1d0-6a191da59ab7 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:07.321233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 9/17] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:07 2022] GET /identity/v3/users/89b57772ff384ce6af49426c38787139 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:18:07.326398 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097af87-b312-420a-b1d0-6a191da59ab7 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:07.340425 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:07.407216 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a00e3f3-a305-40a5-b423-404015f0dafc None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:07.424709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097af87-b312-420a-b1d0-6a191da59ab7 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:07.441734 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ed6b6-f337-4c50-87fe-8715f28cad65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:07.442524 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ed6b6-f337-4c50-87fe-8715f28cad65 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:07.442803 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ed6b6-f337-4c50-87fe-8715f28cad65 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:07.443079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ed6b6-f337-4c50-87fe-8715f28cad65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:07.460757 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f837ba-bdd7-46e8-a9b8-f643f6b788fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:07.461293 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07ed6b6-f337-4c50-87fe-8715f28cad65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:07.461732 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f837ba-bdd7-46e8-a9b8-f643f6b788fa None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:07.462017 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f837ba-bdd7-46e8-a9b8-f643f6b788fa None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:07.462298 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f837ba-bdd7-46e8-a9b8-f643f6b788fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:07.462743 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07ed6b6-f337-4c50-87fe-8715f28cad65 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:07.463490 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f837ba-bdd7-46e8-a9b8-f643f6b788fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:07.464961 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f837ba-bdd7-46e8-a9b8-f643f6b788fa None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:07.466538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 10/18] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:18:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:18:07.472218 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f837ba-bdd7-46e8-a9b8-f643f6b788fa None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:07.486034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-e07ed6b6-f337-4c50-87fe-8715f28cad65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:18:07.486034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:07.486034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:07.486034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:07.486034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:07.486034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:07.486034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:07.486034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:07.486034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:07.486034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:07.486034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:07.486034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:07.486034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:18:07.486034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:18:07.486034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:18:07.487040 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:18:07.487040 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:07.487040 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:07.487040 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:07.487040 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:07.487040 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:07.487040 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:07.487040 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:07.487040 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:18:07.487040 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:07.487040 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:07.487040 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:07.487040 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:07.487040 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:07.487040 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:07.487040 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:07.487040 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:07.488069 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:18:07.488069 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:18:07.488069 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:18:07.488069 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:18:07.488069 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 9/19] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:18:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:18:07.494837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ed6b6-f337-4c50-87fe-8715f28cad65 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:07.551082 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:07.552556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:07.572732 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f837ba-bdd7-46e8-a9b8-f643f6b788fa None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:07.595677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07ed6b6-f337-4c50-87fe-8715f28cad65 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:07.607284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0cbf36-bab5-4484-9548-756f57e4daa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:07.607928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0cbf36-bab5-4484-9548-756f57e4daa3 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:07.608238 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0cbf36-bab5-4484-9548-756f57e4daa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:07.608520 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0cbf36-bab5-4484-9548-756f57e4daa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:07.609345 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0cbf36-bab5-4484-9548-756f57e4daa3 None admin] RBAC: Adding query filter params (role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8, scope.domain.id=default, user.id=89b57772ff384ce6af49426c38787139) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:07.609658 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0cbf36-bab5-4484-9548-756f57e4daa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:07.611107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0cbf36-bab5-4484-9548-756f57e4daa3 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:07.618257 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 11/20] 10.222.0.16 () {56 vars in 1322 bytes} [Mon May 23 12:18:07 2022] GET /identity/v3/role_assignments?user.id=89b57772ff384ce6af49426c38787139&scope.domain.id=default&role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:18:07.629648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe83dce-3e19-408b-a0b2-52be982af6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:07.630337 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe83dce-3e19-408b-a0b2-52be982af6a5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:07.630623 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe83dce-3e19-408b-a0b2-52be982af6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:07.630903 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe83dce-3e19-408b-a0b2-52be982af6a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:07.631686 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe83dce-3e19-408b-a0b2-52be982af6a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:07.631992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe83dce-3e19-408b-a0b2-52be982af6a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:07.633830 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe83dce-3e19-408b-a0b2-52be982af6a5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:07.655500 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 10/21] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:18:10.111054 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad3c99-e447-419e-b2f8-aabe10108c9b None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:10.112075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad3c99-e447-419e-b2f8-aabe10108c9b None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:10.112596 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad3c99-e447-419e-b2f8-aabe10108c9b None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:10.114165 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 12/22] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:10.120502 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9199e6-051e-4a3c-8cb4-ae01ee30fd2d None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:10.121378 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9199e6-051e-4a3c-8cb4-ae01ee30fd2d None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:10.121876 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9199e6-051e-4a3c-8cb4-ae01ee30fd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:10.274314 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-4b9199e6-051e-4a3c-8cb4-ae01ee30fd2d None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:10.275517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-4b9199e6-051e-4a3c-8cb4-ae01ee30fd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:10.366881 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 11/23] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:10.412986 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad3c99-e447-419e-b2f8-aabe10108c9b None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:10.489365 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b751e-84f1-475c-823a-26aacdaa3be8 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:10.489653 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b751e-84f1-475c-823a-26aacdaa3be8 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:10.490169 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b751e-84f1-475c-823a-26aacdaa3be8 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:10.491837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 12/24] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:18:10.497515 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f7343-6ce0-4e77-ba56-1aa6fcc7bb41 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:10.498110 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f7343-6ce0-4e77-ba56-1aa6fcc7bb41 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:10.498610 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f7343-6ce0-4e77-ba56-1aa6fcc7bb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:10.543429 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-467f7343-6ce0-4e77-ba56-1aa6fcc7bb41 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:10.544689 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-467f7343-6ce0-4e77-ba56-1aa6fcc7bb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:10.616972 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad3c99-e447-419e-b2f8-aabe10108c9b None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:10.636198 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:10.683066 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 13/25] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 23 12:18:10.685386 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d06fc-303b-4398-be06-ace12aa01599 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:10.686429 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d06fc-303b-4398-be06-ace12aa01599 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:10.687033 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d06fc-303b-4398-be06-ace12aa01599 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:10.688978 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 14/26] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:10.695031 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7b917-9cb6-470d-ac58-370b56ec8fa9 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:10.696975 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7b917-9cb6-470d-ac58-370b56ec8fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:10.697866 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7b917-9cb6-470d-ac58-370b56ec8fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:10.700776 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb1f408-230f-4f3e-b64c-1f227d94049a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:10.701764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1f408-230f-4f3e-b64c-1f227d94049a None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:10.702252 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1f408-230f-4f3e-b64c-1f227d94049a None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:10.702696 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1f408-230f-4f3e-b64c-1f227d94049a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:10.725341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb1f408-230f-4f3e-b64c-1f227d94049a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:10.736673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-2bb1f408-230f-4f3e-b64c-1f227d94049a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:18:10.736673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:10.736673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:10.736673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:10.736673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:10.736673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:10.736673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:10.736673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:10.736673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:10.736673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:10.736673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:10.736673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:10.736673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:10.736673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:10.736673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:10.736673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:10.737556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:10.737556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:10.737556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:10.737556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:10.737556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:10.737556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:10.737556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:10.737556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:10.737556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:10.737556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:10.737556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:10.737556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:18:10.737556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:10.737556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:10.737556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:10.737556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:10.738326 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:10.738326 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:10.738326 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:10.738326 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:10.738326 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:10.738326 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:10.738326 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:10.738326 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:10.738326 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:10.738326 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:10.738326 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:18:10.738326 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:18:10.738907 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-55c7b917-9cb6-470d-ac58-370b56ec8fa9 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:10.738907 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-55c7b917-9cb6-470d-ac58-370b56ec8fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:10.740864 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 13/27] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:18:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:10.742566 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb1f408-230f-4f3e-b64c-1f227d94049a None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:10.751136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:10.821219 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 15/28] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:10.823696 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30a9eb-31b4-4d1d-a394-ec471767454c None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:10.824041 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30a9eb-31b4-4d1d-a394-ec471767454c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:10.824609 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30a9eb-31b4-4d1d-a394-ec471767454c None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:10.827559 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 16/29] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:10.833031 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30a9eb-31b4-4d1d-a394-ec471767454c None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:10.878079 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb1f408-230f-4f3e-b64c-1f227d94049a None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:10.914379 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad621d3-21ee-4967-8be7-fd353a74dc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:10.915675 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad621d3-21ee-4967-8be7-fd353a74dc44 None admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:10.916163 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad621d3-21ee-4967-8be7-fd353a74dc44 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:10.916654 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad621d3-21ee-4967-8be7-fd353a74dc44 None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:10.917582 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad621d3-21ee-4967-8be7-fd353a74dc44 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:10.919167 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad621d3-21ee-4967-8be7-fd353a74dc44 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:10.959659 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30a9eb-31b4-4d1d-a394-ec471767454c None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:10.973610 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:11.009372 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04083f3-e83e-4816-ad23-7362017cbd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:11.010050 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04083f3-e83e-4816-ad23-7362017cbd88 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:11.010336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04083f3-e83e-4816-ad23-7362017cbd88 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:11.010614 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04083f3-e83e-4816-ad23-7362017cbd88 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:11.011433 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04083f3-e83e-4816-ad23-7362017cbd88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:11.011797 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04083f3-e83e-4816-ad23-7362017cbd88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:11.013220 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04083f3-e83e-4816-ad23-7362017cbd88 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:11.031326 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 17/30] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:18:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:11.031326 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387754c-aa02-4f9d-ba6b-fce9a40cf84f None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:11.031326 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387754c-aa02-4f9d-ba6b-fce9a40cf84f None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:11.031326 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e387754c-aa02-4f9d-ba6b-fce9a40cf84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:11.046956 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-e387754c-aa02-4f9d-ba6b-fce9a40cf84f None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:11.048140 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e387754c-aa02-4f9d-ba6b-fce9a40cf84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:11.056230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 14/31] 10.222.0.16 () {60 vars in 1131 bytes} [Mon May 23 12:18:10 2022] POST /identity/v3/roles => generated 219 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:18:11.057433 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad621d3-21ee-4967-8be7-fd353a74dc44 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:11.204291 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 18/32] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:11.204291 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387754c-aa02-4f9d-ba6b-fce9a40cf84f None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:11.213745 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:11.318377 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad621d3-21ee-4967-8be7-fd353a74dc44 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:11.339518 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:11.344121 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e387754c-aa02-4f9d-ba6b-fce9a40cf84f None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:11.356128 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ea3a3-5203-43f4-9245-7ad85ceabd64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:11.356846 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ea3a3-5203-43f4-9245-7ad85ceabd64 None admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:11.357295 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ea3a3-5203-43f4-9245-7ad85ceabd64 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:11.357723 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ea3a3-5203-43f4-9245-7ad85ceabd64 None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:11.358659 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47ea3a3-5203-43f4-9245-7ad85ceabd64 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:11.360373 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47ea3a3-5203-43f4-9245-7ad85ceabd64 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:11.382388 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 15/33] 10.222.0.16 () {60 vars in 1131 bytes} [Mon May 23 12:18:11 2022] POST /identity/v3/roles => generated 225 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:18:11.384383 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ea3a3-5203-43f4-9245-7ad85ceabd64 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:11.391919 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8834332-bbd4-4e8b-b711-d2abdeeef2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:11.392967 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8834332-bbd4-4e8b-b711-d2abdeeef2c1 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:11.393309 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8834332-bbd4-4e8b-b711-d2abdeeef2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:11.393696 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8834332-bbd4-4e8b-b711-d2abdeeef2c1 None admin] PATH_INFO: `/v3/users/89b57772ff384ce6af49426c38787139` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:11.395628 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8834332-bbd4-4e8b-b711-d2abdeeef2c1 None admin] RBAC: Authorizing `identity:get_user(user_id=89b57772ff384ce6af49426c38787139)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:11.398024 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8834332-bbd4-4e8b-b711-d2abdeeef2c1 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:11.402251 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 19/34] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:11 2022] GET /identity/v3/users/89b57772ff384ce6af49426c38787139 => generated 247 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:11.409421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8834332-bbd4-4e8b-b711-d2abdeeef2c1 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:11.559398 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ea3a3-5203-43f4-9245-7ad85ceabd64 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:11.579366 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8834332-bbd4-4e8b-b711-d2abdeeef2c1 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:11.623627 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4699e7b-b3fb-44dc-b136-c0cda3b4771f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:11.624887 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4699e7b-b3fb-44dc-b136-c0cda3b4771f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:11.625941 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:11.626364 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4699e7b-b3fb-44dc-b136-c0cda3b4771f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:11.627327 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4699e7b-b3fb-44dc-b136-c0cda3b4771f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:11.629244 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4699e7b-b3fb-44dc-b136-c0cda3b4771f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:11.631289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4699e7b-b3fb-44dc-b136-c0cda3b4771f None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:11.635525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 20/35] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:18:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:18:11.642878 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4699e7b-b3fb-44dc-b136-c0cda3b4771f None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:11.646882 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d49ee2-2c67-493b-b993-c1ad9ba2d01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:11.647690 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d49ee2-2c67-493b-b993-c1ad9ba2d01d None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:11.647974 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d49ee2-2c67-493b-b993-c1ad9ba2d01d None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:11.648370 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d49ee2-2c67-493b-b993-c1ad9ba2d01d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:11.660879 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d49ee2-2c67-493b-b993-c1ad9ba2d01d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:11.662376 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d49ee2-2c67-493b-b993-c1ad9ba2d01d None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:11.683264 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-b4d49ee2-2c67-493b-b993-c1ad9ba2d01d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:18:11.683264 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:11.683264 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:11.683264 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:11.683264 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:11.683264 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:11.683264 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:11.683264 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:11.683264 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:11.683264 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:11.683264 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:11.683264 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:11.683264 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:18:11.683264 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:18:11.683264 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:18:11.684156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:18:11.684156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:11.684156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:11.684156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:11.684156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:11.684156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:11.684156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:11.684156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:11.684156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:18:11.684156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:11.684156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:11.684156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:11.684156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:11.684156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:11.684156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:11.684156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:11.684156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:11.685025 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:18:11.685025 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:18:11.685025 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:18:11.685025 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:18:11.686164 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:11.686282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 16/36] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:18:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:11.692802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d49ee2-2c67-493b-b993-c1ad9ba2d01d None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:11.780207 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4699e7b-b3fb-44dc-b136-c0cda3b4771f None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:11.821193 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc123f-6341-47a0-8628-8e767be625d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:11.821992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc123f-6341-47a0-8628-8e767be625d2 None admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:11.822272 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc123f-6341-47a0-8628-8e767be625d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:11.822548 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc123f-6341-47a0-8628-8e767be625d2 None admin] PATH_INFO: `/v3/domains/default/users/89b57772ff384ce6af49426c38787139/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:11.824677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fc123f-6341-47a0-8628-8e767be625d2 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=89b57772ff384ce6af49426c38787139, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:11.826140 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fc123f-6341-47a0-8628-8e767be625d2 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:11.841282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d49ee2-2c67-493b-b993-c1ad9ba2d01d None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:11.881557 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a60a90-2121-4546-b525-d954c46f0806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:11.882230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a60a90-2121-4546-b525-d954c46f0806 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:11.882516 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a60a90-2121-4546-b525-d954c46f0806 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:11.883324 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a60a90-2121-4546-b525-d954c46f0806 None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:11.884582 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a60a90-2121-4546-b525-d954c46f0806 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:11.884897 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a60a90-2121-4546-b525-d954c46f0806 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:11.886346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a60a90-2121-4546-b525-d954c46f0806 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:11.907168 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 17/37] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:18:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:11.912509 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a60a90-2121-4546-b525-d954c46f0806 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:11.951688 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:11.957277 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 21/38] 10.222.0.16 () {58 vars in 1361 bytes} [Mon May 23 12:18:11 2022] PUT /identity/v3/domains/default/users/89b57772ff384ce6af49426c38787139/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:18:11.959043 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed2de2-d79b-4c32-bc62-e326bd7d188e None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:11.959343 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed2de2-d79b-4c32-bc62-e326bd7d188e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:11.959630 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed2de2-d79b-4c32-bc62-e326bd7d188e None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:11.964687 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 22/39] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:11.970348 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1f161-7999-402b-b26f-a54a8904f7fa None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:11.970645 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1f161-7999-402b-b26f-a54a8904f7fa None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:11.970928 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1f161-7999-402b-b26f-a54a8904f7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:11.989964 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-03e1f161-7999-402b-b26f-a54a8904f7fa None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:11.991234 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-03e1f161-7999-402b-b26f-a54a8904f7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:12.044912 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a60a90-2121-4546-b525-d954c46f0806 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:12.072799 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 23/40] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 12:18:12.074083 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef3087-ba8e-4739-aa27-a2af896bfeb2 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:12.074372 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef3087-ba8e-4739-aa27-a2af896bfeb2 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:12.074656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef3087-ba8e-4739-aa27-a2af896bfeb2 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:12.075963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 24/41] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:12.082007 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a5416-e7e6-4ad4-83f7-c2edc115f6d8 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:12.082367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a5416-e7e6-4ad4-83f7-c2edc115f6d8 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:12.082774 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368a5416-e7e6-4ad4-83f7-c2edc115f6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:12.091702 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de38ab-3b82-447c-8379-537939f21dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:12.092655 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de38ab-3b82-447c-8379-537939f21dfa None admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:12.093088 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de38ab-3b82-447c-8379-537939f21dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:12.093580 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de38ab-3b82-447c-8379-537939f21dfa None admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:12.094492 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10de38ab-3b82-447c-8379-537939f21dfa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:12.096255 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10de38ab-3b82-447c-8379-537939f21dfa None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:12.104477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-368a5416-e7e6-4ad4-83f7-c2edc115f6d8 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:12.105739 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-368a5416-e7e6-4ad4-83f7-c2edc115f6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:12.127215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:12.132363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 18/42] 10.222.0.16 () {60 vars in 1141 bytes} [Mon May 23 12:18:11 2022] POST /identity/v3/projects => generated 301 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:18:12.132363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de38ab-3b82-447c-8379-537939f21dfa None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:12.201904 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 25/43] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 12:18:12.229672 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a5416-e7e6-4ad4-83f7-c2edc115f6d8 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:12.247555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:12.270783 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de38ab-3b82-447c-8379-537939f21dfa None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:12.326256 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b39ecf-985b-4681-b970-ea00db57b9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:12.328138 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b39ecf-985b-4681-b970-ea00db57b9aa None admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:12.328606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b39ecf-985b-4681-b970-ea00db57b9aa None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:12.329059 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b39ecf-985b-4681-b970-ea00db57b9aa None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:12.329992 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b39ecf-985b-4681-b970-ea00db57b9aa None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:12.331574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b39ecf-985b-4681-b970-ea00db57b9aa None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:12.355644 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 19/44] 10.222.0.16 () {60 vars in 1131 bytes} [Mon May 23 12:18:12 2022] POST /identity/v3/roles => generated 223 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:18:12.364523 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368a5416-e7e6-4ad4-83f7-c2edc115f6d8 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:12.412315 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ceaafd-8f04-49f8-b5ef-33b060ae80f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:12.413148 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ceaafd-8f04-49f8-b5ef-33b060ae80f6 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:12.413440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ceaafd-8f04-49f8-b5ef-33b060ae80f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:12.413727 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ceaafd-8f04-49f8-b5ef-33b060ae80f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:12.415445 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ceaafd-8f04-49f8-b5ef-33b060ae80f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:12.417143 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ceaafd-8f04-49f8-b5ef-33b060ae80f6 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:12.419134 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 26/45] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:18:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:18:12.424867 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b39ecf-985b-4681-b970-ea00db57b9aa None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:12.542595 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b39ecf-985b-4681-b970-ea00db57b9aa None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:12.583464 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034e34e-974b-4eea-ba1a-9b5d357e842e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:12.588135 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034e34e-974b-4eea-ba1a-9b5d357e842e None admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:12.588504 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034e34e-974b-4eea-ba1a-9b5d357e842e None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:12.588799 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034e34e-974b-4eea-ba1a-9b5d357e842e None admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:12.589548 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2034e34e-974b-4eea-ba1a-9b5d357e842e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:12.591051 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2034e34e-974b-4eea-ba1a-9b5d357e842e None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:12.613509 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:12.620433 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 20/46] 10.222.0.16 () {60 vars in 1141 bytes} [Mon May 23 12:18:12 2022] POST /identity/v3/projects => generated 312 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:18:14.224753 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab56a73-a035-4782-9783-0eb6b3965007 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:14.225292 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab56a73-a035-4782-9783-0eb6b3965007 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:14.225392 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab56a73-a035-4782-9783-0eb6b3965007 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:14.226757 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 27/47] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:14.232197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b17490-0187-47d5-aea8-15bac28d7b73 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:14.232578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b17490-0187-47d5-aea8-15bac28d7b73 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:14.232796 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b17490-0187-47d5-aea8-15bac28d7b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:14.249689 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-76b17490-0187-47d5-aea8-15bac28d7b73 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:14.251276 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-76b17490-0187-47d5-aea8-15bac28d7b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:14.317583 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 21/48] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:14.341970 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab56a73-a035-4782-9783-0eb6b3965007 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:14.366231 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:14.457004 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab56a73-a035-4782-9783-0eb6b3965007 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:14.503786 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b6c2e-ccef-4356-bf37-3556f1b73a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:14.504614 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b6c2e-ccef-4356-bf37-3556f1b73a05 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:14.504901 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b6c2e-ccef-4356-bf37-3556f1b73a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:14.505182 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b6c2e-ccef-4356-bf37-3556f1b73a05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:14.517531 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b6c2e-ccef-4356-bf37-3556f1b73a05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:14.526223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-1d4b6c2e-ccef-4356-bf37-3556f1b73a05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:18:14.526223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:14.526223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:14.526223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:14.526223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:14.526223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:14.526223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:14.526223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:14.526223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:14.526223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:14.526223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:14.526223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:14.526223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:14.526223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:14.526223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:14.526223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:14.527103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:14.527103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:14.527103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:14.527103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:14.527103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:14.527103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:14.527103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:14.527103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:14.527103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:14.527103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:14.527103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:14.527103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:18:14.527103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:14.527103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:14.527103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:14.527103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:14.527881 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:14.527881 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:14.527881 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:14.527881 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:14.527881 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:14.527881 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:14.527881 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:14.527881 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:14.527881 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:14.527881 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:14.527881 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:18:14.527881 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:18:14.527881 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 28/49] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:18:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:14.533495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b17490-0187-47d5-aea8-15bac28d7b73 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:14.639033 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b17490-0187-47d5-aea8-15bac28d7b73 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:14.668098 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:14.679214 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d9c77-c507-4845-83b8-e28c6656c804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:14.680659 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d9c77-c507-4845-83b8-e28c6656c804 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:14.681049 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d9c77-c507-4845-83b8-e28c6656c804 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:14.681506 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d9c77-c507-4845-83b8-e28c6656c804 None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:14.682332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d9c77-c507-4845-83b8-e28c6656c804 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:14.682651 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d9c77-c507-4845-83b8-e28c6656c804 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:14.684104 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d9c77-c507-4845-83b8-e28c6656c804 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:14.699336 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 22/50] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:18:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:14.708876 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b6c2e-ccef-4356-bf37-3556f1b73a05 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:14.811310 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:14.832491 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b6c2e-ccef-4356-bf37-3556f1b73a05 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:14.868016 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356752f-a035-4e12-be7b-a0c6f759f3ed None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:14.868512 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356752f-a035-4e12-be7b-a0c6f759f3ed None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:14.869291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356752f-a035-4e12-be7b-a0c6f759f3ed None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:14.870910 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 23/51] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:14.872293 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a36cd-fa63-44d7-a62d-e035099fd029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:14.873178 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a36cd-fa63-44d7-a62d-e035099fd029 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:14.873465 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a36cd-fa63-44d7-a62d-e035099fd029 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:14.873747 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a36cd-fa63-44d7-a62d-e035099fd029 None admin] PATH_INFO: `/v3/users/89b57772ff384ce6af49426c38787139` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:14.875038 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a36cd-fa63-44d7-a62d-e035099fd029 None admin] RBAC: Authorizing `identity:get_user(user_id=89b57772ff384ce6af49426c38787139)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:14.876109 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d806465-2c2c-40fe-854c-4d03afc22553 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:14.876447 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d806465-2c2c-40fe-854c-4d03afc22553 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:14.876687 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a36cd-fa63-44d7-a62d-e035099fd029 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:14.876806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d806465-2c2c-40fe-854c-4d03afc22553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:14.878417 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 29/52] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:14 2022] GET /identity/v3/users/89b57772ff384ce6af49426c38787139 => generated 247 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:14.883839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a36cd-fa63-44d7-a62d-e035099fd029 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:14.894904 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-9d806465-2c2c-40fe-854c-4d03afc22553 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:14.896038 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-9d806465-2c2c-40fe-854c-4d03afc22553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:14.966394 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 24/53] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 12:18:14.991279 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d806465-2c2c-40fe-854c-4d03afc22553 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:15.000364 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a36cd-fa63-44d7-a62d-e035099fd029 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:15.040733 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b168754-803f-4133-904b-492ba3ac1cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:15.041578 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b168754-803f-4133-904b-492ba3ac1cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:15.041877 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b168754-803f-4133-904b-492ba3ac1cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:15.042164 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b168754-803f-4133-904b-492ba3ac1cb1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:15.043416 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b168754-803f-4133-904b-492ba3ac1cb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:15.044958 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b168754-803f-4133-904b-492ba3ac1cb1 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:15.046833 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 30/54] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:18:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:18:15.054270 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b168754-803f-4133-904b-492ba3ac1cb1 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:15.074681 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:15.098405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d806465-2c2c-40fe-854c-4d03afc22553 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:15.124761 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:15.137724 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d882dc-becd-4038-84df-c33f771d04e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:15.138531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d882dc-becd-4038-84df-c33f771d04e2 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:15.138816 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d882dc-becd-4038-84df-c33f771d04e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:15.139098 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d882dc-becd-4038-84df-c33f771d04e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:15.140408 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d882dc-becd-4038-84df-c33f771d04e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:15.148772 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d882dc-becd-4038-84df-c33f771d04e2 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:15.150087 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 25/55] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:18:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:18:15.155397 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d882dc-becd-4038-84df-c33f771d04e2 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:15.168620 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b168754-803f-4133-904b-492ba3ac1cb1 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:15.205701 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd1b50-99ae-404f-8088-c97318a683e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:15.206379 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd1b50-99ae-404f-8088-c97318a683e1 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:15.206674 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd1b50-99ae-404f-8088-c97318a683e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:15.206962 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd1b50-99ae-404f-8088-c97318a683e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:15.207809 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cd1b50-99ae-404f-8088-c97318a683e1 None admin] RBAC: Adding query filter params (role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8, scope.domain.id=default, user.id=89b57772ff384ce6af49426c38787139) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:15.208129 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cd1b50-99ae-404f-8088-c97318a683e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:15.209630 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cd1b50-99ae-404f-8088-c97318a683e1 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:15.218459 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 31/56] 10.222.0.16 () {56 vars in 1322 bytes} [Mon May 23 12:18:15 2022] GET /identity/v3/role_assignments?user.id=89b57772ff384ce6af49426c38787139&scope.domain.id=default&role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 530 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:18:15.278707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d882dc-becd-4038-84df-c33f771d04e2 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:15.323282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd090cf-25cf-4598-ae85-e630b44c07de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:15.324206 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd090cf-25cf-4598-ae85-e630b44c07de None admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:15.324498 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd090cf-25cf-4598-ae85-e630b44c07de None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:15.324780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd090cf-25cf-4598-ae85-e630b44c07de None admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:15.325528 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd090cf-25cf-4598-ae85-e630b44c07de None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:15.327079 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd090cf-25cf-4598-ae85-e630b44c07de None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:15.358816 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 26/57] 10.222.0.16 () {60 vars in 1140 bytes} [Mon May 23 12:18:15 2022] POST /identity/v3/projects => generated 298 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:18:17.577045 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d73a91-dfd9-4ea0-b95b-7e685479fb03 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:17.577595 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d73a91-dfd9-4ea0-b95b-7e685479fb03 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:17.577595 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d73a91-dfd9-4ea0-b95b-7e685479fb03 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:17.578950 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 32/58] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:17.584013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aba6ab-f6e4-40ac-ab9f-2581f121396c None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:17.584419 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aba6ab-f6e4-40ac-ab9f-2581f121396c None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:17.584620 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aba6ab-f6e4-40ac-ab9f-2581f121396c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:17.602699 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-f7aba6ab-f6e4-40ac-ab9f-2581f121396c None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:17.603900 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-f7aba6ab-f6e4-40ac-ab9f-2581f121396c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:17.637174 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:17.669831 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 27/59] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:17.693630 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d73a91-dfd9-4ea0-b95b-7e685479fb03 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:17.775178 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:17.794296 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d73a91-dfd9-4ea0-b95b-7e685479fb03 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:17.826729 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718da82-ef56-4fe5-be98-f33d6dd72eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:17.827709 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718da82-ef56-4fe5-be98-f33d6dd72eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:17.827948 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718da82-ef56-4fe5-be98-f33d6dd72eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:17.828261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718da82-ef56-4fe5-be98-f33d6dd72eb7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:17.829435 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718da82-ef56-4fe5-be98-f33d6dd72eb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:17.830875 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718da82-ef56-4fe5-be98-f33d6dd72eb7 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:17.832518 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 33/60] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:18:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:17.838896 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aba6ab-f6e4-40ac-ab9f-2581f121396c None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:17.936995 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aba6ab-f6e4-40ac-ab9f-2581f121396c None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:17.971406 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25ebe6-6533-4848-bf9e-ec863cdecfa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:17.972080 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25ebe6-6533-4848-bf9e-ec863cdecfa5 None admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:17.972401 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25ebe6-6533-4848-bf9e-ec863cdecfa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:17.972689 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25ebe6-6533-4848-bf9e-ec863cdecfa5 None admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:17.973406 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25ebe6-6533-4848-bf9e-ec863cdecfa5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:17.974861 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25ebe6-6533-4848-bf9e-ec863cdecfa5 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:18.000035 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 28/61] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 12:18:17 2022] POST /identity/v3/users => generated 275 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:18:21.281321 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24ecc0-42d7-40d5-be1e-3749dd3db7c0 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:21.281809 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24ecc0-42d7-40d5-be1e-3749dd3db7c0 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:21.281809 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24ecc0-42d7-40d5-be1e-3749dd3db7c0 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:21.283244 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 34/62] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:21.289075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9fd54-ca0b-47d3-8ac6-c3ad966ab567 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:21.289941 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9fd54-ca0b-47d3-8ac6-c3ad966ab567 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:21.290385 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9fd54-ca0b-47d3-8ac6-c3ad966ab567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:21.314072 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-5af9fd54-ca0b-47d3-8ac6-c3ad966ab567 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:21.315516 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-5af9fd54-ca0b-47d3-8ac6-c3ad966ab567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:21.346359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:21.407528 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 29/63] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 12:18:21.439040 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24ecc0-42d7-40d5-be1e-3749dd3db7c0 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:21.503442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbecb57-33f2-46fc-86de-7e2194f70883 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:21.503769 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbecb57-33f2-46fc-86de-7e2194f70883 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:21.504013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbecb57-33f2-46fc-86de-7e2194f70883 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:21.506215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 30/64] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:21.519070 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524331bb-8113-4978-8c03-910c3bb29a67 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:21.519922 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524331bb-8113-4978-8c03-910c3bb29a67 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:21.520443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524331bb-8113-4978-8c03-910c3bb29a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:21.550082 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-524331bb-8113-4978-8c03-910c3bb29a67 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:21.551630 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-524331bb-8113-4978-8c03-910c3bb29a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:21.583035 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24ecc0-42d7-40d5-be1e-3749dd3db7c0 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:21.643693 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 31/65] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:21.664909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df917c3-c27d-472c-b212-312bddd12c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:21.665731 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df917c3-c27d-472c-b212-312bddd12c13 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:21.666024 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df917c3-c27d-472c-b212-312bddd12c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:21.666312 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df917c3-c27d-472c-b212-312bddd12c13 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:21.682201 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df917c3-c27d-472c-b212-312bddd12c13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:21.684562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932755af-a4ff-4e05-91d4-613f2f8641d4 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:21.685269 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932755af-a4ff-4e05-91d4-613f2f8641d4 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:21.685821 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932755af-a4ff-4e05-91d4-613f2f8641d4 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:21.687395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 32/66] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:18:21.688323 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932755af-a4ff-4e05-91d4-613f2f8641d4 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:21.690719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-6df917c3-c27d-472c-b212-312bddd12c13 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 12:18:21.690719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:21.690719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:21.690719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:21.690719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:21.690719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:21.690719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:21.690719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:21.690719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:21.690719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:21.690719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:21.690719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:21.690719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:21.690719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:21.690719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:21.690719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:21.691920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:21.691920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:21.691920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:21.691920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:21.691920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:21.691920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:21.691920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:21.691920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:21.691920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:21.691920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:21.691920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:21.691920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:18:21.691920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:21.691920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:21.691920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:21.691920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:21.692755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:21.692755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:21.692755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:21.692755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:21.692755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:21.692755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:21.692755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:21.692755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:21.692755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:21.692755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:21.692755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 12:18:21.692755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:18:21.692755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 35/67] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:18:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:21.693894 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6aa2de-b4e4-4e6b-b0ea-adeb639b3a7b None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:21.693998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6aa2de-b4e4-4e6b-b0ea-adeb639b3a7b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:21.694232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6aa2de-b4e4-4e6b-b0ea-adeb639b3a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:21.719286 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-5f6aa2de-b4e4-4e6b-b0ea-adeb639b3a7b None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:21.719891 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5f6aa2de-b4e4-4e6b-b0ea-adeb639b3a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:21.723313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:21.802014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 36/68] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:21.802874 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6aa2de-b4e4-4e6b-b0ea-adeb639b3a7b None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:21.804043 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:21.831727 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932755af-a4ff-4e05-91d4-613f2f8641d4 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:21.870460 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f65fc-baf5-4ceb-97ff-368ca2de7f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:21.871642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f65fc-baf5-4ceb-97ff-368ca2de7f21 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:21.872292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f65fc-baf5-4ceb-97ff-368ca2de7f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:21.872780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f65fc-baf5-4ceb-97ff-368ca2de7f21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:21.885630 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f65fc-baf5-4ceb-97ff-368ca2de7f21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:21.894312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-f81f65fc-baf5-4ceb-97ff-368ca2de7f21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:18:21.894312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:21.894312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:21.894312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:21.894312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:21.894312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:21.894312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:21.894312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:21.894312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:21.894312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:21.894312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:21.894312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:21.894312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:21.894312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:21.894312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:21.894312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:21.895188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:21.895188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:21.895188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:21.895188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:21.895188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:21.895188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:21.895188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:21.895188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:21.895188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:21.895188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:21.895188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:21.895188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:18:21.895188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:21.895188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:21.895188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:21.895188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:21.895947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:21.895947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:21.895947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:21.895947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:21.895947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:21.895947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:21.895947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:21.895947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:21.895947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:21.895947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:21.895947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:18:21.895947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:18:21.898558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 33/69] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:18:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:21.899653 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f65fc-baf5-4ceb-97ff-368ca2de7f21 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:21.909260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6aa2de-b4e4-4e6b-b0ea-adeb639b3a7b None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:21.946644 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643cf25-4d71-4e33-8e98-63f7c273b0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:21.947331 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643cf25-4d71-4e33-8e98-63f7c273b0af None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:21.947604 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643cf25-4d71-4e33-8e98-63f7c273b0af None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:21.947884 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1643cf25-4d71-4e33-8e98-63f7c273b0af None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:21.948688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1643cf25-4d71-4e33-8e98-63f7c273b0af None admin] RBAC: Adding query filter params (name=member) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:21.949000 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1643cf25-4d71-4e33-8e98-63f7c273b0af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:21.950394 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1643cf25-4d71-4e33-8e98-63f7c273b0af None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:21.958916 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 37/70] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 12:18:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:21.964995 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643cf25-4d71-4e33-8e98-63f7c273b0af None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:21.967932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:22.010917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f65fc-baf5-4ceb-97ff-368ca2de7f21 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:22.048248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12299414-da82-4c28-994c-572172ffdd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:22.049073 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12299414-da82-4c28-994c-572172ffdd81 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:22.049367 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12299414-da82-4c28-994c-572172ffdd81 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:22.049657 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12299414-da82-4c28-994c-572172ffdd81 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:22.060685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12299414-da82-4c28-994c-572172ffdd81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:22.069474 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-12299414-da82-4c28-994c-572172ffdd81 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 12:18:22.069474 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:22.069474 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:22.069474 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:22.069474 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:22.069474 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:22.069474 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:22.069474 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:22.069474 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:22.069474 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:22.069474 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:22.069474 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:22.069474 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:22.069474 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:22.069474 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:22.069474 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:22.070806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:22.070806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:22.070806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:22.070806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:22.070806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:22.070806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:22.070806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:22.070806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:22.070806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:22.070806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:22.070806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:22.070806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:18:22.070806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:22.070806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:22.070806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:22.070806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:22.071701 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:22.071701 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:22.071701 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:22.071701 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:22.071701 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:22.071701 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:22.071701 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:22.071701 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:22.071701 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:22.071701 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:22.071701 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 12:18:22.071701 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:18:22.071701 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 34/71] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 12:18:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:22.076985 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9fbda-5819-47c1-ac97-28008f280289 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:22.077287 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9fbda-5819-47c1-ac97-28008f280289 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:22.077568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9fbda-5819-47c1-ac97-28008f280289 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:22.078755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1643cf25-4d71-4e33-8e98-63f7c273b0af None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:22.083749 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 35/72] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:22.083749 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9fbda-5819-47c1-ac97-28008f280289 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:22.095360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:22.121772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc6074-c8c7-4a3a-bc8d-d67f65a36d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:22.122691 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc6074-c8c7-4a3a-bc8d-d67f65a36d11 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:22.123132 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc6074-c8c7-4a3a-bc8d-d67f65a36d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:22.123425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc6074-c8c7-4a3a-bc8d-d67f65a36d11 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:22.124213 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc6074-c8c7-4a3a-bc8d-d67f65a36d11 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:22.124530 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc6074-c8c7-4a3a-bc8d-d67f65a36d11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:22.125925 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc6074-c8c7-4a3a-bc8d-d67f65a36d11 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:22.135225 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 38/73] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:18:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:22.136045 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc6074-c8c7-4a3a-bc8d-d67f65a36d11 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:22.202538 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:22.206284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9fbda-5819-47c1-ac97-28008f280289 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:22.245404 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc6074-c8c7-4a3a-bc8d-d67f65a36d11 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:22.246890 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fbc36e-ff52-4c2c-a455-e09926ec1cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:22.248847 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fbc36e-ff52-4c2c-a455-e09926ec1cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:22.249157 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fbc36e-ff52-4c2c-a455-e09926ec1cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:22.249446 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fbc36e-ff52-4c2c-a455-e09926ec1cd9 None admin] PATH_INFO: `/v3/users/66b6b05f6db5476b801ab332f119cbdd` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:22.264344 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fbc36e-ff52-4c2c-a455-e09926ec1cd9 None admin] RBAC: Authorizing `identity:get_user(user_id=66b6b05f6db5476b801ab332f119cbdd)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:22.265838 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fbc36e-ff52-4c2c-a455-e09926ec1cd9 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:22.271996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 36/74] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:22 2022] GET /identity/v3/users/66b6b05f6db5476b801ab332f119cbdd => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:22.271996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e52ba6-2fe4-4753-b7ee-522614498c13 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:22.271996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e52ba6-2fe4-4753-b7ee-522614498c13 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:22.271996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e52ba6-2fe4-4753-b7ee-522614498c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:22.286047 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-c1e52ba6-2fe4-4753-b7ee-522614498c13 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:22.287246 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-c1e52ba6-2fe4-4753-b7ee-522614498c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:22.287392 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb49f4-a2a5-439a-8689-dff8d08b94d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:22.288075 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb49f4-a2a5-439a-8689-dff8d08b94d0 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:22.288400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb49f4-a2a5-439a-8689-dff8d08b94d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:22.289315 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb49f4-a2a5-439a-8689-dff8d08b94d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:22.290082 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beb49f4-a2a5-439a-8689-dff8d08b94d0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:22.290397 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beb49f4-a2a5-439a-8689-dff8d08b94d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:22.291784 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beb49f4-a2a5-439a-8689-dff8d08b94d0 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:22.299975 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 39/75] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:18:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:22.300678 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb49f4-a2a5-439a-8689-dff8d08b94d0 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:22.351417 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:22.359496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 37/76] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:22.360252 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e52ba6-2fe4-4753-b7ee-522614498c13 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:22.401572 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb49f4-a2a5-439a-8689-dff8d08b94d0 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:22.426116 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:22.434926 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c4141-019c-47fb-b2f8-c9892bda2c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:22.435805 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c4141-019c-47fb-b2f8-c9892bda2c51 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:22.436095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c4141-019c-47fb-b2f8-c9892bda2c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:22.436410 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c4141-019c-47fb-b2f8-c9892bda2c51 None admin] PATH_INFO: `/v3/users/89b57772ff384ce6af49426c38787139` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:22.437620 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1c4141-019c-47fb-b2f8-c9892bda2c51 None admin] RBAC: Authorizing `identity:get_user(user_id=89b57772ff384ce6af49426c38787139)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:22.439065 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1c4141-019c-47fb-b2f8-c9892bda2c51 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:22.440724 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 40/77] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:22 2022] GET /identity/v3/users/89b57772ff384ce6af49426c38787139 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:22.442143 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64934046-7e7d-4f13-89a2-09c0e07a9784 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:22.442436 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64934046-7e7d-4f13-89a2-09c0e07a9784 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:22.442731 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64934046-7e7d-4f13-89a2-09c0e07a9784 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:22.443946 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 41/78] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:22.448307 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64934046-7e7d-4f13-89a2-09c0e07a9784 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:22.463819 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e52ba6-2fe4-4753-b7ee-522614498c13 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:22.497629 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da0f0a-239a-4e50-9c66-b07c8969c932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:22.498278 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da0f0a-239a-4e50-9c66-b07c8969c932 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:22.498579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da0f0a-239a-4e50-9c66-b07c8969c932 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:22.498846 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da0f0a-239a-4e50-9c66-b07c8969c932 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:22.546240 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64934046-7e7d-4f13-89a2-09c0e07a9784 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:22.564664 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:22.573159 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da0f0a-239a-4e50-9c66-b07c8969c932 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:22.574693 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da0f0a-239a-4e50-9c66-b07c8969c932 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:22.580386 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8b7fb-d49c-4317-b97e-8070b95fa0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:22.581260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8b7fb-d49c-4317-b97e-8070b95fa0ce None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:22.581551 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8b7fb-d49c-4317-b97e-8070b95fa0ce None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:22.581835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8b7fb-d49c-4317-b97e-8070b95fa0ce None admin] PATH_INFO: `/v3/users/66b6b05f6db5476b801ab332f119cbdd` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:22.583025 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f8b7fb-d49c-4317-b97e-8070b95fa0ce None admin] RBAC: Authorizing `identity:get_user(user_id=66b6b05f6db5476b801ab332f119cbdd)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:22.584489 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f8b7fb-d49c-4317-b97e-8070b95fa0ce None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:22.587171 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 42/79] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:22 2022] GET /identity/v3/users/66b6b05f6db5476b801ab332f119cbdd => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:22.587862 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8b7fb-d49c-4317-b97e-8070b95fa0ce None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:22.650329 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:22.673828 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 38/80] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:18:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:18:22.674680 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da0f0a-239a-4e50-9c66-b07c8969c932 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:22.689688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8b7fb-d49c-4317-b97e-8070b95fa0ce None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:22.723621 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a0158-1ffd-4706-83c6-77e0e3aa16fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:22.724450 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a0158-1ffd-4706-83c6-77e0e3aa16fd None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:22.724749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a0158-1ffd-4706-83c6-77e0e3aa16fd None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:22.725033 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a0158-1ffd-4706-83c6-77e0e3aa16fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:22.735682 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0a0158-1ffd-4706-83c6-77e0e3aa16fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:22.743898 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-8f0a0158-1ffd-4706-83c6-77e0e3aa16fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 12:18:22.743898 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:22.743898 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:22.743898 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:22.743898 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:22.743898 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:22.743898 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:22.743898 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:22.743898 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:22.743898 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:22.743898 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:22.743898 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:22.743898 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:22.743898 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:22.743898 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:22.743898 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:22.745017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:22.745017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:22.745017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:22.745017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:22.745017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:22.745017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:22.745017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:22.745017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:22.745017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:22.745017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:22.745017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:22.745017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:18:22.745017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:22.745017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:22.745017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:22.745017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:22.745988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:22.745988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:22.745988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:22.745988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:22.745988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:22.745988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:22.745988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:22.745988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:22.745988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:22.745988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:22.745988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 12:18:22.745988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:18:22.745988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 43/81] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:18:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:22.746688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91008845-47cf-4900-8efb-e04c8975e66f None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:22.746887 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91008845-47cf-4900-8efb-e04c8975e66f None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:22.747173 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91008845-47cf-4900-8efb-e04c8975e66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:22.763348 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-91008845-47cf-4900-8efb-e04c8975e66f None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:22.764579 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-91008845-47cf-4900-8efb-e04c8975e66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:22.769255 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:22.776342 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da0f0a-239a-4e50-9c66-b07c8969c932 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:22.811298 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60a288-1711-40d7-ad38-c0a44f5ea43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:22.811945 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60a288-1711-40d7-ad38-c0a44f5ea43a None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:22.812251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60a288-1711-40d7-ad38-c0a44f5ea43a None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:22.812533 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60a288-1711-40d7-ad38-c0a44f5ea43a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:22.830595 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 44/82] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:22.831441 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91008845-47cf-4900-8efb-e04c8975e66f None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:22.884270 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60a288-1711-40d7-ad38-c0a44f5ea43a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:22.885766 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60a288-1711-40d7-ad38-c0a44f5ea43a None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:22.887103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:22.934335 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91008845-47cf-4900-8efb-e04c8975e66f None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:22.967638 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef705329-ed5e-47ee-adf1-96a8d138441c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:22.968314 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef705329-ed5e-47ee-adf1-96a8d138441c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:22.968607 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef705329-ed5e-47ee-adf1-96a8d138441c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:22.968889 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef705329-ed5e-47ee-adf1-96a8d138441c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:22.978202 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:22.985641 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 39/83] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:18:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:18:22.986495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60a288-1711-40d7-ad38-c0a44f5ea43a None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:23.040695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef705329-ed5e-47ee-adf1-96a8d138441c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:23.042187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef705329-ed5e-47ee-adf1-96a8d138441c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:23.090845 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60a288-1711-40d7-ad38-c0a44f5ea43a None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:23.094071 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:23.124372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe225059-1910-4811-8cb0-2a1b37e933bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:23.125233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe225059-1910-4811-8cb0-2a1b37e933bf None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:23.125520 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe225059-1910-4811-8cb0-2a1b37e933bf None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:23.125805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe225059-1910-4811-8cb0-2a1b37e933bf None admin] PATH_INFO: `/v3/projects/2bb553c7fe6e4284b56d30c136b7a041` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:23.126995 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe225059-1910-4811-8cb0-2a1b37e933bf None admin] RBAC: Authorizing `identity:get_project(project_id=2bb553c7fe6e4284b56d30c136b7a041)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:23.128479 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe225059-1910-4811-8cb0-2a1b37e933bf None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:23.130163 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 40/84] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:18:22 2022] GET /identity/v3/projects/2bb553c7fe6e4284b56d30c136b7a041 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:23.130933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe225059-1910-4811-8cb0-2a1b37e933bf None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:23.141618 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 45/85] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:18:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:18:23.142451 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef705329-ed5e-47ee-adf1-96a8d138441c None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:23.199801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:23.237116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe225059-1910-4811-8cb0-2a1b37e933bf None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:23.238218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef705329-ed5e-47ee-adf1-96a8d138441c None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:23.270768 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057623da-403d-4814-9f63-a70b8dc4d29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:23.271430 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057623da-403d-4814-9f63-a70b8dc4d29c None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:23.271554 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a504d-29a8-4205-9760-0772b786beaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:23.271724 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057623da-403d-4814-9f63-a70b8dc4d29c None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:23.272034 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057623da-403d-4814-9f63-a70b8dc4d29c None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:23.272282 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a504d-29a8-4205-9760-0772b786beaf None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:23.272573 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a504d-29a8-4205-9760-0772b786beaf None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:23.272831 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057623da-403d-4814-9f63-a70b8dc4d29c None admin] RBAC: Adding query filter params (name=member) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:23.273533 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a504d-29a8-4205-9760-0772b786beaf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:23.273693 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057623da-403d-4814-9f63-a70b8dc4d29c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:23.274582 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a504d-29a8-4205-9760-0772b786beaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:23.275101 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057623da-403d-4814-9f63-a70b8dc4d29c None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:23.276037 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a504d-29a8-4205-9760-0772b786beaf None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:23.277672 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 46/86] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:18:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:23.278429 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a504d-29a8-4205-9760-0772b786beaf None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:23.290412 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 41/87] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 12:18:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:23.290412 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057623da-403d-4814-9f63-a70b8dc4d29c None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:23.299428 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:23.377680 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a504d-29a8-4205-9760-0772b786beaf None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:23.386329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057623da-403d-4814-9f63-a70b8dc4d29c None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:23.411666 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:23.412334 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d88878d-05d7-49c4-936e-8bcaf93f86aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:23.413012 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88878d-05d7-49c4-936e-8bcaf93f86aa None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:23.413384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88878d-05d7-49c4-936e-8bcaf93f86aa None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:23.413763 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88878d-05d7-49c4-936e-8bcaf93f86aa None admin] PATH_INFO: `/v3/projects/2bb553c7fe6e4284b56d30c136b7a041` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:23.415057 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d88878d-05d7-49c4-936e-8bcaf93f86aa None admin] RBAC: Authorizing `identity:get_project(project_id=2bb553c7fe6e4284b56d30c136b7a041)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:23.416609 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d88878d-05d7-49c4-936e-8bcaf93f86aa None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:23.418479 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 47/88] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:18:23 2022] GET /identity/v3/projects/2bb553c7fe6e4284b56d30c136b7a041 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:23.419128 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d88878d-05d7-49c4-936e-8bcaf93f86aa None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:23.421614 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686f59d-f7d4-418b-bd95-9fc43445b1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:23.422274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686f59d-f7d4-418b-bd95-9fc43445b1be None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:23.422568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686f59d-f7d4-418b-bd95-9fc43445b1be None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:23.422853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686f59d-f7d4-418b-bd95-9fc43445b1be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:23.423692 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686f59d-f7d4-418b-bd95-9fc43445b1be None admin] RBAC: Adding query filter params (role.id=504a45d998f0464893c90fa5429275a8, scope.project.id=2bb553c7fe6e4284b56d30c136b7a041, user.id=66b6b05f6db5476b801ab332f119cbdd) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:23.424018 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686f59d-f7d4-418b-bd95-9fc43445b1be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:23.425510 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686f59d-f7d4-418b-bd95-9fc43445b1be None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:23.431413 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 42/89] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:18:23 2022] GET /identity/v3/role_assignments?user.id=66b6b05f6db5476b801ab332f119cbdd&scope.project.id=2bb553c7fe6e4284b56d30c136b7a041&role.id=504a45d998f0464893c90fa5429275a8 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:23.432203 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686f59d-f7d4-418b-bd95-9fc43445b1be None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:23.518814 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:23.538693 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d88878d-05d7-49c4-936e-8bcaf93f86aa None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:23.559459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686f59d-f7d4-418b-bd95-9fc43445b1be None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:23.579139 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b58d13-a9ca-44a7-aecb-7320a5b92071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:23.580117 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b58d13-a9ca-44a7-aecb-7320a5b92071 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:23.580443 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b58d13-a9ca-44a7-aecb-7320a5b92071 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:23.580728 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b58d13-a9ca-44a7-aecb-7320a5b92071 None admin] PATH_INFO: `/v3/projects/2bb553c7fe6e4284b56d30c136b7a041` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:23.581956 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b58d13-a9ca-44a7-aecb-7320a5b92071 None admin] RBAC: Authorizing `identity:get_project(project_id=2bb553c7fe6e4284b56d30c136b7a041)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:23.583447 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b58d13-a9ca-44a7-aecb-7320a5b92071 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:23.585485 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 48/90] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:18:23 2022] GET /identity/v3/projects/2bb553c7fe6e4284b56d30c136b7a041 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:23.586126 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b58d13-a9ca-44a7-aecb-7320a5b92071 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:23.603222 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ee33f-5216-4cef-a328-84b92b8d33e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:23.604249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ee33f-5216-4cef-a328-84b92b8d33e7 None admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:23.604542 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ee33f-5216-4cef-a328-84b92b8d33e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:23.604825 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94ee33f-5216-4cef-a328-84b92b8d33e7 None admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:23.605553 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ee33f-5216-4cef-a328-84b92b8d33e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:23.607042 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94ee33f-5216-4cef-a328-84b92b8d33e7 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:23.631633 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 43/91] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 12:18:23 2022] POST /identity/v3/users => generated 289 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:18:23.633478 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ee33f-5216-4cef-a328-84b92b8d33e7 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:23.720591 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:23.724840 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b58d13-a9ca-44a7-aecb-7320a5b92071 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:23.768635 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe650461-26d1-4a62-a632-fe2336d741ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:23.769804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe650461-26d1-4a62-a632-fe2336d741ae None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:23.770084 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe650461-26d1-4a62-a632-fe2336d741ae None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:23.770460 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe650461-26d1-4a62-a632-fe2336d741ae None admin] PATH_INFO: `/v3/users/66b6b05f6db5476b801ab332f119cbdd` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:23.771901 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe650461-26d1-4a62-a632-fe2336d741ae None admin] RBAC: Authorizing `identity:get_user(user_id=66b6b05f6db5476b801ab332f119cbdd)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:23.774410 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe650461-26d1-4a62-a632-fe2336d741ae None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:23.776234 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94ee33f-5216-4cef-a328-84b92b8d33e7 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:23.776667 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 49/92] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:23 2022] GET /identity/v3/users/66b6b05f6db5476b801ab332f119cbdd => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:23.777558 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe650461-26d1-4a62-a632-fe2336d741ae None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:23.810224 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:23.829243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e960c4-01a5-4a8e-a9b2-25a5662c6f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:23.829922 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e960c4-01a5-4a8e-a9b2-25a5662c6f1f None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:23.830217 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e960c4-01a5-4a8e-a9b2-25a5662c6f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:23.830507 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e960c4-01a5-4a8e-a9b2-25a5662c6f1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:23.831980 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e960c4-01a5-4a8e-a9b2-25a5662c6f1f None admin] RBAC: Adding query filter params (role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8, scope.project.id=2bb553c7fe6e4284b56d30c136b7a041, user.id=89b57772ff384ce6af49426c38787139) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:23.832328 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e960c4-01a5-4a8e-a9b2-25a5662c6f1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:23.834313 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e960c4-01a5-4a8e-a9b2-25a5662c6f1f None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:23.842003 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 44/93] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:18:23 2022] GET /identity/v3/role_assignments?user.id=89b57772ff384ce6af49426c38787139&scope.project.id=2bb553c7fe6e4284b56d30c136b7a041&role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:23.849144 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e960c4-01a5-4a8e-a9b2-25a5662c6f1f None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:23.913050 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe650461-26d1-4a62-a632-fe2336d741ae None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:23.961893 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f69e8f-df23-48cd-9dc8-d0cc808409a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:23.962911 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f69e8f-df23-48cd-9dc8-d0cc808409a1 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:23.963390 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f69e8f-df23-48cd-9dc8-d0cc808409a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:23.964423 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f69e8f-df23-48cd-9dc8-d0cc808409a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:23.965495 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f69e8f-df23-48cd-9dc8-d0cc808409a1 None admin] RBAC: Adding query filter params (role.id=2d4732dacad6444e86d7fee3514afefb, scope.project.id=2bb553c7fe6e4284b56d30c136b7a041, user.id=66b6b05f6db5476b801ab332f119cbdd) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:23.966329 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f69e8f-df23-48cd-9dc8-d0cc808409a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:23.968447 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f69e8f-df23-48cd-9dc8-d0cc808409a1 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:23.977770 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 50/94] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:18:23 2022] GET /identity/v3/role_assignments?user.id=66b6b05f6db5476b801ab332f119cbdd&scope.project.id=2bb553c7fe6e4284b56d30c136b7a041&role.id=2d4732dacad6444e86d7fee3514afefb => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:24.009280 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e960c4-01a5-4a8e-a9b2-25a5662c6f1f None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:24.029521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:24.063647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92e6a0-5004-4031-91d1-22e50651ccaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:24.064349 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92e6a0-5004-4031-91d1-22e50651ccaa None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:24.064835 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92e6a0-5004-4031-91d1-22e50651ccaa None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:24.065125 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92e6a0-5004-4031-91d1-22e50651ccaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:24.149724 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d92e6a0-5004-4031-91d1-22e50651ccaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:24.151320 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d92e6a0-5004-4031-91d1-22e50651ccaa None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:24.265893 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 45/95] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:18:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:18:24.271295 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f69e8f-df23-48cd-9dc8-d0cc808409a1 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:24.373520 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:24.387553 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f69e8f-df23-48cd-9dc8-d0cc808409a1 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:24.428964 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169a858-dbe3-446c-bba4-13e4be9574ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:24.430099 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169a858-dbe3-446c-bba4-13e4be9574ce None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:24.430448 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169a858-dbe3-446c-bba4-13e4be9574ce None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:24.430744 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169a858-dbe3-446c-bba4-13e4be9574ce None admin] PATH_INFO: `/v3/projects/139a085a71c14dee9992a865eefb6a88` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:24.432265 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7169a858-dbe3-446c-bba4-13e4be9574ce None admin] RBAC: Authorizing `identity:get_project(project_id=139a085a71c14dee9992a865eefb6a88)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:24.433783 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7169a858-dbe3-446c-bba4-13e4be9574ce None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:24.435838 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 51/96] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:18:24 2022] GET /identity/v3/projects/139a085a71c14dee9992a865eefb6a88 => generated 312 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:24.441760 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92e6a0-5004-4031-91d1-22e50651ccaa None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:24.454282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:24.562236 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92e6a0-5004-4031-91d1-22e50651ccaa None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:24.606523 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76729cf4-f8d6-4026-9df5-a4a847eaa74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:24.607190 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76729cf4-f8d6-4026-9df5-a4a847eaa74a None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:24.607477 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76729cf4-f8d6-4026-9df5-a4a847eaa74a None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:24.607816 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76729cf4-f8d6-4026-9df5-a4a847eaa74a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:24.608690 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76729cf4-f8d6-4026-9df5-a4a847eaa74a None admin] RBAC: Adding query filter params (role.id=504a45d998f0464893c90fa5429275a8, scope.project.id=139a085a71c14dee9992a865eefb6a88, user.id=66b6b05f6db5476b801ab332f119cbdd) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:24.609047 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76729cf4-f8d6-4026-9df5-a4a847eaa74a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:24.610500 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76729cf4-f8d6-4026-9df5-a4a847eaa74a None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:24.620255 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 46/97] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:18:24 2022] GET /identity/v3/role_assignments?user.id=66b6b05f6db5476b801ab332f119cbdd&scope.project.id=139a085a71c14dee9992a865eefb6a88&role.id=504a45d998f0464893c90fa5429275a8 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:26.258705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a54f65-724f-4600-9317-45d4734c615e None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:26.259367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a54f65-724f-4600-9317-45d4734c615e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:26.259367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a54f65-724f-4600-9317-45d4734c615e None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:26.260682 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 52/98] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:26.266144 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77672885-90da-44d7-b90f-bb2cb0d0dafc None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:26.266578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77672885-90da-44d7-b90f-bb2cb0d0dafc None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:26.266755 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77672885-90da-44d7-b90f-bb2cb0d0dafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:26.288654 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-77672885-90da-44d7-b90f-bb2cb0d0dafc None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:26.290169 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-77672885-90da-44d7-b90f-bb2cb0d0dafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:26.391657 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 47/99] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 12:18:26.446341 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a54f65-724f-4600-9317-45d4734c615e None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:26.579791 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a54f65-724f-4600-9317-45d4734c615e None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:26.628186 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82065eca-e638-4475-a637-ab847580b0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:26.628996 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82065eca-e638-4475-a637-ab847580b0eb None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:26.629288 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82065eca-e638-4475-a637-ab847580b0eb None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:26.629576 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82065eca-e638-4475-a637-ab847580b0eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:26.649229 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82065eca-e638-4475-a637-ab847580b0eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:26.657019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-82065eca-e638-4475-a637-ab847580b0eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 12:18:26.657019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:26.657019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:26.657019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:26.657019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:26.657019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:26.657019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:26.657019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:26.657019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:26.657019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:26.657019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:26.657019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:26.657019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:26.657019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:26.657019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:26.657019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:26.657925 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:26.657925 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:26.657925 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:26.657925 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:26.657925 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:26.657925 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:26.657925 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:26.657925 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:26.657925 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:26.657925 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:26.657925 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:26.657925 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:18:26.657925 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:26.657925 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:26.657925 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:26.657925 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:26.659087 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:26.659087 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:26.659087 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:26.659087 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:26.659087 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:26.659087 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:26.659087 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:26.659087 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:26.659087 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:26.659087 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:26.659087 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 12:18:26.659087 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:18:26.659087 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 53/100] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:18:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:26.666569 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77672885-90da-44d7-b90f-bb2cb0d0dafc None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:26.674196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:26.823767 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dcaf8a-dea2-414e-aa80-aa143f3423a1 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:26.824934 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dcaf8a-dea2-414e-aa80-aa143f3423a1 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:26.825456 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dcaf8a-dea2-414e-aa80-aa143f3423a1 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:26.827032 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 54/101] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:18:26.833746 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435292e-d4d2-4605-bd3c-7403eb40ff92 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:26.833986 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435292e-d4d2-4605-bd3c-7403eb40ff92 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:26.834274 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435292e-d4d2-4605-bd3c-7403eb40ff92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:26.857169 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77672885-90da-44d7-b90f-bb2cb0d0dafc None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:26.881222 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-9435292e-d4d2-4605-bd3c-7403eb40ff92 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:26.883086 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-9435292e-d4d2-4605-bd3c-7403eb40ff92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:26.899942 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4e355-f981-49f3-b897-60458a4487ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:26.903529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4e355-f981-49f3-b897-60458a4487ac None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:26.904036 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4e355-f981-49f3-b897-60458a4487ac None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:26.904546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4e355-f981-49f3-b897-60458a4487ac None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:26.905587 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4e355-f981-49f3-b897-60458a4487ac None admin] RBAC: Adding query filter params (name=member) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:26.906102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4e355-f981-49f3-b897-60458a4487ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:26.907800 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4e355-f981-49f3-b897-60458a4487ac None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:26.909779 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:26.924196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 48/102] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 12:18:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:26.926932 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4e355-f981-49f3-b897-60458a4487ac None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:27.001064 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:27.042991 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4e355-f981-49f3-b897-60458a4487ac None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:27.081322 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd3cc1-ede4-446a-8013-7e6f3ff2f20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:27.082562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd3cc1-ede4-446a-8013-7e6f3ff2f20d None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:27.083054 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd3cc1-ede4-446a-8013-7e6f3ff2f20d None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:27.083534 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdd3cc1-ede4-446a-8013-7e6f3ff2f20d None admin] PATH_INFO: `/v3/users/66b6b05f6db5476b801ab332f119cbdd` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:27.085086 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd3cc1-ede4-446a-8013-7e6f3ff2f20d None admin] RBAC: Authorizing `identity:get_user(user_id=66b6b05f6db5476b801ab332f119cbdd)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:27.088225 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdd3cc1-ede4-446a-8013-7e6f3ff2f20d None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:27.095733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 49/103] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:26 2022] GET /identity/v3/users/66b6b05f6db5476b801ab332f119cbdd => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:27.097888 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd3cc1-ede4-446a-8013-7e6f3ff2f20d None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:27.098666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 55/104] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 12:18:27.201673 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435292e-d4d2-4605-bd3c-7403eb40ff92 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:27.233937 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdd3cc1-ede4-446a-8013-7e6f3ff2f20d None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:27.284430 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a10bf8-2397-405a-b6aa-a4af8eb50b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:27.286208 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a10bf8-2397-405a-b6aa-a4af8eb50b8e None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:27.286700 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a10bf8-2397-405a-b6aa-a4af8eb50b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:27.287209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a10bf8-2397-405a-b6aa-a4af8eb50b8e None admin] PATH_INFO: `/v3/projects/2bb553c7fe6e4284b56d30c136b7a041` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:27.288753 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a10bf8-2397-405a-b6aa-a4af8eb50b8e None admin] RBAC: Authorizing `identity:get_project(project_id=2bb553c7fe6e4284b56d30c136b7a041)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:27.290504 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a10bf8-2397-405a-b6aa-a4af8eb50b8e None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:27.292485 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 50/105] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:18:27 2022] GET /identity/v3/projects/2bb553c7fe6e4284b56d30c136b7a041 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:18:27.297494 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e44eb-1be2-46ad-854a-6c980c2e184f None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:27.298050 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e44eb-1be2-46ad-854a-6c980c2e184f None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:27.298534 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e44eb-1be2-46ad-854a-6c980c2e184f None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:27.300098 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 51/106] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:27.302906 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e44eb-1be2-46ad-854a-6c980c2e184f None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:27.306817 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:27.420825 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e44eb-1be2-46ad-854a-6c980c2e184f None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:27.458004 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c1c312-718c-4559-8797-5316fe667b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:27.459269 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1c312-718c-4559-8797-5316fe667b0f None admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:27.459780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1c312-718c-4559-8797-5316fe667b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:27.460850 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c1c312-718c-4559-8797-5316fe667b0f None admin] PATH_INFO: `/v3/projects/2bb553c7fe6e4284b56d30c136b7a041/users/66b6b05f6db5476b801ab332f119cbdd/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:27.463490 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1c312-718c-4559-8797-5316fe667b0f None admin] RBAC: Authorizing `identity:create_grant(project_id=2bb553c7fe6e4284b56d30c136b7a041, user_id=66b6b05f6db5476b801ab332f119cbdd, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:27.464808 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435292e-d4d2-4605-bd3c-7403eb40ff92 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:27.465947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c1c312-718c-4559-8797-5316fe667b0f None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:27.490479 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 52/107] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 12:18:27 2022] PUT /identity/v3/projects/2bb553c7fe6e4284b56d30c136b7a041/users/66b6b05f6db5476b801ab332f119cbdd/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:18:27.492682 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e626d-410a-4ec6-a626-feeef7f47376 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:27.493230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e626d-410a-4ec6-a626-feeef7f47376 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:27.493714 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e626d-410a-4ec6-a626-feeef7f47376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:27.526570 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-b02e626d-410a-4ec6-a626-feeef7f47376 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:27.528126 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-b02e626d-410a-4ec6-a626-feeef7f47376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:27.560743 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d09ae0-b941-49c5-a2fc-33c6a41b5bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:27.562003 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d09ae0-b941-49c5-a2fc-33c6a41b5bda None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:27.562342 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d09ae0-b941-49c5-a2fc-33c6a41b5bda None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:27.562635 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d09ae0-b941-49c5-a2fc-33c6a41b5bda None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:27.597008 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d09ae0-b941-49c5-a2fc-33c6a41b5bda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:27.617666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-d0d09ae0-b941-49c5-a2fc-33c6a41b5bda None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 12:18:27.617666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:27.617666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:27.617666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:27.617666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:27.617666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:27.617666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:27.617666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:27.617666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:27.617666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:27.617666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:27.617666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:27.617666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:27.617666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:27.617666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:27.617666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:27.619102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:27.619102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:27.619102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:27.619102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:27.619102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:27.619102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:27.619102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:27.619102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:27.619102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:27.619102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:27.619102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:27.619102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:18:27.619102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:27.619102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:27.619102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:27.619102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:27.620097 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:27.620097 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:27.620097 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:27.620097 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:27.620097 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:27.620097 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:27.620097 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:27.620097 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:27.620097 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:27.620097 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:27.620097 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 12:18:27.620097 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:18:27.620097 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 56/108] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 12:18:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 419 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:27.633864 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d09ae0-b941-49c5-a2fc-33c6a41b5bda None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:27.650797 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:27.705647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:27.729636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 53/109] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:27.777384 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e626d-410a-4ec6-a626-feeef7f47376 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:27.829389 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d09ae0-b941-49c5-a2fc-33c6a41b5bda None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:27.879393 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb9fd0-43d0-4f3e-a9f6-571c3f104316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:27.880380 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb9fd0-43d0-4f3e-a9f6-571c3f104316 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:27.880678 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb9fd0-43d0-4f3e-a9f6-571c3f104316 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:27.881220 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb9fd0-43d0-4f3e-a9f6-571c3f104316 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:27.882570 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb9fd0-43d0-4f3e-a9f6-571c3f104316 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:27.883165 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb9fd0-43d0-4f3e-a9f6-571c3f104316 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:27.885256 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb9fd0-43d0-4f3e-a9f6-571c3f104316 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:27.896673 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 57/110] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:18:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:27.898571 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28222b94-773f-4af2-9ffb-5e8ceafd3ecd None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:27.898866 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28222b94-773f-4af2-9ffb-5e8ceafd3ecd None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:27.899152 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28222b94-773f-4af2-9ffb-5e8ceafd3ecd None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:27.900988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 58/111] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:27.902510 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b813c2-200d-4247-86a4-ff26dd6ef1d7 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:27.903103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b813c2-200d-4247-86a4-ff26dd6ef1d7 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:27.903393 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b813c2-200d-4247-86a4-ff26dd6ef1d7 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:27.908690 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 59/112] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:27.909451 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b813c2-200d-4247-86a4-ff26dd6ef1d7 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:27.939764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e626d-410a-4ec6-a626-feeef7f47376 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:27.985873 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bfc79f-136b-442a-8b13-440fd3069bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:27.986590 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfc79f-136b-442a-8b13-440fd3069bdb None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:27.986877 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfc79f-136b-442a-8b13-440fd3069bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:27.987160 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfc79f-136b-442a-8b13-440fd3069bdb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:27.988609 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:28.000353 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bfc79f-136b-442a-8b13-440fd3069bdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:28.008056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-32bfc79f-136b-442a-8b13-440fd3069bdb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:18:28.008056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:28.008056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:28.008056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:28.008056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:28.008056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:28.008056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:28.008056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:28.008056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:28.008056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:28.008056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:28.008056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:28.008056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:28.008056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:28.008056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:28.008056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:28.008987 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:28.008987 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:28.008987 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:28.008987 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:28.008987 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:28.008987 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:28.008987 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:28.008987 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:28.008987 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:28.008987 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:28.008987 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:28.008987 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:18:28.008987 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:28.008987 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:28.008987 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:28.008987 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:28.009896 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:28.009896 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:28.009896 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:28.009896 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:28.009896 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:28.009896 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:28.009896 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:28.009896 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:28.009896 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:28.009896 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:28.009896 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:18:28.009896 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:18:28.009896 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 54/113] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:18:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:18:28.010817 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26160d43-a360-4e6b-a5b5-a6b922bd3b02 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:28.011112 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26160d43-a360-4e6b-a5b5-a6b922bd3b02 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:28.011400 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26160d43-a360-4e6b-a5b5-a6b922bd3b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:28.033357 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b813c2-200d-4247-86a4-ff26dd6ef1d7 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:28.034899 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-26160d43-a360-4e6b-a5b5-a6b922bd3b02 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:28.036097 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-26160d43-a360-4e6b-a5b5-a6b922bd3b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:28.071866 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a531081-161d-4165-8858-48ff2312fb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:28.072786 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a531081-161d-4165-8858-48ff2312fb01 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:28.073079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a531081-161d-4165-8858-48ff2312fb01 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:28.073373 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a531081-161d-4165-8858-48ff2312fb01 None admin] PATH_INFO: `/v3/users/66b6b05f6db5476b801ab332f119cbdd` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:28.074577 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a531081-161d-4165-8858-48ff2312fb01 None admin] RBAC: Authorizing `identity:get_user(user_id=66b6b05f6db5476b801ab332f119cbdd)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:28.076042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a531081-161d-4165-8858-48ff2312fb01 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:28.077712 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 60/114] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:27 2022] GET /identity/v3/users/66b6b05f6db5476b801ab332f119cbdd => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:28.079011 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da43fa0-13dd-4f9e-974f-9aa1c1f09658 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:28.079305 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da43fa0-13dd-4f9e-974f-9aa1c1f09658 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:28.079589 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da43fa0-13dd-4f9e-974f-9aa1c1f09658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:28.099109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-4da43fa0-13dd-4f9e-974f-9aa1c1f09658 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:28.100331 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-4da43fa0-13dd-4f9e-974f-9aa1c1f09658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:28.102025 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:28.122948 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 55/115] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:28.124299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b66ec6-2b11-434b-b368-a634f09d51ca None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:28.124595 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b66ec6-2b11-434b-b368-a634f09d51ca None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:28.124878 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b66ec6-2b11-434b-b368-a634f09d51ca None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:28.126284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 56/116] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:28.130810 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b66ec6-2b11-434b-b368-a634f09d51ca None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:28.183775 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 61/117] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:28.185584 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da43fa0-13dd-4f9e-974f-9aa1c1f09658 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:28.242073 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b66ec6-2b11-434b-b368-a634f09d51ca None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:28.259764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:28.279399 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e3c77-cac8-4093-8f2b-dfea74fb553c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:28.280074 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e3c77-cac8-4093-8f2b-dfea74fb553c None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:28.280392 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e3c77-cac8-4093-8f2b-dfea74fb553c None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:28.280677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e3c77-cac8-4093-8f2b-dfea74fb553c None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:28.281440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168e3c77-cac8-4093-8f2b-dfea74fb553c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:28.281754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168e3c77-cac8-4093-8f2b-dfea74fb553c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:28.283162 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168e3c77-cac8-4093-8f2b-dfea74fb553c None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:28.291900 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 57/118] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:18:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:28.293184 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c901c2b-15c7-439e-8409-8b627e2180f7 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:28.293477 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c901c2b-15c7-439e-8409-8b627e2180f7 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:28.293757 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c901c2b-15c7-439e-8409-8b627e2180f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:28.302711 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da43fa0-13dd-4f9e-974f-9aa1c1f09658 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:28.310924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-0c901c2b-15c7-439e-8409-8b627e2180f7 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:28.312112 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-0c901c2b-15c7-439e-8409-8b627e2180f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:28.340324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f40bdcc-933b-4914-bb17-c7878a72d7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:28.341187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f40bdcc-933b-4914-bb17-c7878a72d7f2 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:28.341477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f40bdcc-933b-4914-bb17-c7878a72d7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:28.341761 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f40bdcc-933b-4914-bb17-c7878a72d7f2 None admin] PATH_INFO: `/v3/projects/2bb553c7fe6e4284b56d30c136b7a041` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:28.342935 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f40bdcc-933b-4914-bb17-c7878a72d7f2 None admin] RBAC: Authorizing `identity:get_project(project_id=2bb553c7fe6e4284b56d30c136b7a041)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:28.344399 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f40bdcc-933b-4914-bb17-c7878a72d7f2 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:28.350640 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 62/119] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:18:28 2022] GET /identity/v3/projects/2bb553c7fe6e4284b56d30c136b7a041 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:28.350640 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f40bdcc-933b-4914-bb17-c7878a72d7f2 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:28.360665 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:28.391090 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 58/120] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:28.391614 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c901c2b-15c7-439e-8409-8b627e2180f7 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:28.474031 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f40bdcc-933b-4914-bb17-c7878a72d7f2 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:28.499243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c901c2b-15c7-439e-8409-8b627e2180f7 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:28.517172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d8d2f-e690-4756-b442-5671bf7b8037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:28.518359 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d8d2f-e690-4756-b442-5671bf7b8037 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:28.518651 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d8d2f-e690-4756-b442-5671bf7b8037 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:28.518933 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d8d2f-e690-4756-b442-5671bf7b8037 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:28.521479 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:28.532874 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841d8d2f-e690-4756-b442-5671bf7b8037 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:28.536288 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08b98d-2896-4787-8977-0e0e1592633c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:28.537099 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08b98d-2896-4787-8977-0e0e1592633c None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:28.537389 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08b98d-2896-4787-8977-0e0e1592633c None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:28.537673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08b98d-2896-4787-8977-0e0e1592633c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:28.540811 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-841d8d2f-e690-4756-b442-5671bf7b8037 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 12:18:28.540811 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:28.540811 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:28.540811 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:28.540811 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:28.540811 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:28.540811 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:28.540811 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:28.540811 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:28.540811 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:28.540811 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:28.540811 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:28.540811 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:28.540811 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:28.540811 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:28.540811 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:28.541665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:28.541665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:28.541665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:28.541665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:28.541665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:28.541665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:28.541665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:28.541665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:28.541665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:28.541665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:28.541665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:28.541665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:18:28.541665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:28.541665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:28.541665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:28.541665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:28.542421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:28.542421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:28.542421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:28.542421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:28.542421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:28.542421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:28.542421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:28.542421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:28.542421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:28.542421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:28.542421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 12:18:28.542421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:18:28.542421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 63/121] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:18:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:28.543067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d8d2f-e690-4756-b442-5671bf7b8037 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:28.554867 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da08b98d-2896-4787-8977-0e0e1592633c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:28.563383 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-da08b98d-2896-4787-8977-0e0e1592633c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 12:18:28.563383 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:28.563383 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:28.563383 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:28.563383 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:28.563383 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:28.563383 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:28.563383 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:28.563383 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:28.563383 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:28.563383 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:28.563383 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:28.563383 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:28.563383 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:28.563383 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:28.563383 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:28.564265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:28.564265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:28.564265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:28.564265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:28.564265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:28.564265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:28.564265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:28.564265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:28.564265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:28.564265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:28.564265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:28.564265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:18:28.564265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:28.564265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:28.564265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:28.564265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:28.565079 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:28.565079 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:28.565079 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:28.565079 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:28.565079 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:28.565079 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:28.565079 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:28.565079 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:28.565079 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:28.565079 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:28.565079 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 12:18:28.565079 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:18:28.565079 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 59/122] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:18:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:28.565722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08b98d-2896-4787-8977-0e0e1592633c None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:28.607902 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:28.653909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d8d2f-e690-4756-b442-5671bf7b8037 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:28.677478 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08b98d-2896-4787-8977-0e0e1592633c None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:28.692582 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e874e0-7493-4abe-ba7c-34437e56acc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:28.693725 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e874e0-7493-4abe-ba7c-34437e56acc4 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:28.694082 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e874e0-7493-4abe-ba7c-34437e56acc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:28.694457 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e874e0-7493-4abe-ba7c-34437e56acc4 None admin] PATH_INFO: `/v3/users/89b57772ff384ce6af49426c38787139` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:28.696142 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e874e0-7493-4abe-ba7c-34437e56acc4 None admin] RBAC: Authorizing `identity:get_user(user_id=89b57772ff384ce6af49426c38787139)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:28.697630 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e874e0-7493-4abe-ba7c-34437e56acc4 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:28.699415 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 64/123] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:28 2022] GET /identity/v3/users/89b57772ff384ce6af49426c38787139 => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:28.700014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e874e0-7493-4abe-ba7c-34437e56acc4 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:28.717695 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f584d9-006b-4dbb-ba3f-0df40fab9fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:28.718596 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f584d9-006b-4dbb-ba3f-0df40fab9fa0 None admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:28.719040 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f584d9-006b-4dbb-ba3f-0df40fab9fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:28.719206 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f584d9-006b-4dbb-ba3f-0df40fab9fa0 None admin] PATH_INFO: `/v3/projects/2bb553c7fe6e4284b56d30c136b7a041/users/66b6b05f6db5476b801ab332f119cbdd/roles/2d4732dacad6444e86d7fee3514afefb` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:28.721445 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f584d9-006b-4dbb-ba3f-0df40fab9fa0 None admin] RBAC: Authorizing `identity:create_grant(project_id=2bb553c7fe6e4284b56d30c136b7a041, user_id=66b6b05f6db5476b801ab332f119cbdd, role_id=2d4732dacad6444e86d7fee3514afefb)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:28.723037 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f584d9-006b-4dbb-ba3f-0df40fab9fa0 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:28.740284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 60/124] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 12:18:28 2022] PUT /identity/v3/projects/2bb553c7fe6e4284b56d30c136b7a041/users/66b6b05f6db5476b801ab332f119cbdd/roles/2d4732dacad6444e86d7fee3514afefb => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:18:28.741148 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f584d9-006b-4dbb-ba3f-0df40fab9fa0 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:28.769578 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:28.820275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e874e0-7493-4abe-ba7c-34437e56acc4 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:28.873010 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e26de7-074a-4393-b379-723d3108128d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:28.874300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e26de7-074a-4393-b379-723d3108128d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:28.874596 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e26de7-074a-4393-b379-723d3108128d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:28.874880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e26de7-074a-4393-b379-723d3108128d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:28.876490 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e26de7-074a-4393-b379-723d3108128d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:28.878834 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e26de7-074a-4393-b379-723d3108128d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:28.881066 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 65/125] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:18:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:28.885237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e26de7-074a-4393-b379-723d3108128d None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:28.904292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:28.916647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f584d9-006b-4dbb-ba3f-0df40fab9fa0 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:28.973287 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b660ed3-da5a-482d-a169-42b1537d2d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:28.974944 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b660ed3-da5a-482d-a169-42b1537d2d78 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:28.975576 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b660ed3-da5a-482d-a169-42b1537d2d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:28.976760 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b660ed3-da5a-482d-a169-42b1537d2d78 None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:28.977634 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b660ed3-da5a-482d-a169-42b1537d2d78 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:28.977954 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b660ed3-da5a-482d-a169-42b1537d2d78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:28.980289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b660ed3-da5a-482d-a169-42b1537d2d78 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:28.990956 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 61/126] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 12:18:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:28.991839 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b660ed3-da5a-482d-a169-42b1537d2d78 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:29.024665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e26de7-074a-4393-b379-723d3108128d None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:29.060888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:29.069438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa736716-5fbd-4053-bd9c-4887f0e0e0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:29.071107 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa736716-5fbd-4053-bd9c-4887f0e0e0f9 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:29.071535 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa736716-5fbd-4053-bd9c-4887f0e0e0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:29.072155 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa736716-5fbd-4053-bd9c-4887f0e0e0f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:29.073425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa736716-5fbd-4053-bd9c-4887f0e0e0f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:29.074141 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa736716-5fbd-4053-bd9c-4887f0e0e0f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:29.075664 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa736716-5fbd-4053-bd9c-4887f0e0e0f9 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:29.087376 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 66/127] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 12:18:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:29.092636 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa736716-5fbd-4053-bd9c-4887f0e0e0f9 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:29.126574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b660ed3-da5a-482d-a169-42b1537d2d78 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:29.168383 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f85747-619b-4eb3-9d5f-39a10f23a973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:29.169549 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f85747-619b-4eb3-9d5f-39a10f23a973 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:29.169841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f85747-619b-4eb3-9d5f-39a10f23a973 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:29.170281 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f85747-619b-4eb3-9d5f-39a10f23a973 None admin] PATH_INFO: `/v3/projects/2bb553c7fe6e4284b56d30c136b7a041` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:29.172024 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f85747-619b-4eb3-9d5f-39a10f23a973 None admin] RBAC: Authorizing `identity:get_project(project_id=2bb553c7fe6e4284b56d30c136b7a041)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:29.174156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f85747-619b-4eb3-9d5f-39a10f23a973 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:29.176544 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 62/128] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:18:28 2022] GET /identity/v3/projects/2bb553c7fe6e4284b56d30c136b7a041 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:29.177526 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f85747-619b-4eb3-9d5f-39a10f23a973 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:29.215775 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:29.228308 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa736716-5fbd-4053-bd9c-4887f0e0e0f9 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:29.268722 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80ae66-71f2-41c6-b6ac-4d918d807933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:29.269401 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80ae66-71f2-41c6-b6ac-4d918d807933 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:29.270185 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80ae66-71f2-41c6-b6ac-4d918d807933 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:29.270478 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80ae66-71f2-41c6-b6ac-4d918d807933 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:29.271488 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac80ae66-71f2-41c6-b6ac-4d918d807933 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:29.273228 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac80ae66-71f2-41c6-b6ac-4d918d807933 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:29.311347 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 67/129] 10.222.0.16 () {60 vars in 1141 bytes} [Mon May 23 12:18:29 2022] POST /identity/v3/projects => generated 302 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:18:29.312983 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80ae66-71f2-41c6-b6ac-4d918d807933 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:29.315900 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f85747-619b-4eb3-9d5f-39a10f23a973 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:29.390942 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017785c-07a9-4551-9d94-dd719a867f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:29.392286 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017785c-07a9-4551-9d94-dd719a867f55 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:29.393716 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017785c-07a9-4551-9d94-dd719a867f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:29.394183 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017785c-07a9-4551-9d94-dd719a867f55 None admin] PATH_INFO: `/v3/users/1f94383c11514c06966b43e65c85d8f3` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:29.419185 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:29.427734 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2017785c-07a9-4551-9d94-dd719a867f55 None admin] RBAC: Authorizing `identity:get_user(user_id=1f94383c11514c06966b43e65c85d8f3)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:29.432693 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2017785c-07a9-4551-9d94-dd719a867f55 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:29.435815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 63/130] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:29 2022] GET /identity/v3/users/1f94383c11514c06966b43e65c85d8f3 => generated 289 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:29.437100 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017785c-07a9-4551-9d94-dd719a867f55 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:29.555036 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:29.572686 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80ae66-71f2-41c6-b6ac-4d918d807933 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:29.581842 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017785c-07a9-4551-9d94-dd719a867f55 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:29.613443 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc3e17-667b-4ff9-9a79-3435057074ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:29.614683 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc3e17-667b-4ff9-9a79-3435057074ca None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:29.614978 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc3e17-667b-4ff9-9a79-3435057074ca None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:29.615356 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cc3e17-667b-4ff9-9a79-3435057074ca None admin] PATH_INFO: `/v3/users/66b6b05f6db5476b801ab332f119cbdd` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:29.617328 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc3e17-667b-4ff9-9a79-3435057074ca None admin] RBAC: Authorizing `identity:get_user(user_id=66b6b05f6db5476b801ab332f119cbdd)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:29.619242 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cc3e17-667b-4ff9-9a79-3435057074ca None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:29.624217 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 68/131] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:29 2022] GET /identity/v3/users/66b6b05f6db5476b801ab332f119cbdd => generated 275 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:29.624217 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc3e17-667b-4ff9-9a79-3435057074ca None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:29.627901 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc686b90-6687-458b-9181-1b7115aecef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:29.628846 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc686b90-6687-458b-9181-1b7115aecef8 None admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:29.629134 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc686b90-6687-458b-9181-1b7115aecef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:29.629409 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc686b90-6687-458b-9181-1b7115aecef8 None admin] PATH_INFO: `/v3/projects/2bb553c7fe6e4284b56d30c136b7a041/users/89b57772ff384ce6af49426c38787139/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:29.634550 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc686b90-6687-458b-9181-1b7115aecef8 None admin] RBAC: Authorizing `identity:create_grant(project_id=2bb553c7fe6e4284b56d30c136b7a041, user_id=89b57772ff384ce6af49426c38787139, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:29.636953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc686b90-6687-458b-9181-1b7115aecef8 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:29.655048 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 64/132] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 12:18:29 2022] PUT /identity/v3/projects/2bb553c7fe6e4284b56d30c136b7a041/users/89b57772ff384ce6af49426c38787139/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:18:29.655918 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc686b90-6687-458b-9181-1b7115aecef8 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:29.795191 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cc3e17-667b-4ff9-9a79-3435057074ca None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:29.827487 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:29.850528 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc686b90-6687-458b-9181-1b7115aecef8 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:29.870171 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7299d6a5-d6f5-4683-8360-3010006e6b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:29.870839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299d6a5-d6f5-4683-8360-3010006e6b9c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:29.871123 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299d6a5-d6f5-4683-8360-3010006e6b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:29.871401 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7299d6a5-d6f5-4683-8360-3010006e6b9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:29.927764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d9171-c91c-4f3f-a212-cfd25ff73a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:29.928697 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d9171-c91c-4f3f-a212-cfd25ff73a84 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:29.929042 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d9171-c91c-4f3f-a212-cfd25ff73a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:29.929831 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5d9171-c91c-4f3f-a212-cfd25ff73a84 None admin] PATH_INFO: `/v3/projects/139a085a71c14dee9992a865eefb6a88` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:29.932571 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d9171-c91c-4f3f-a212-cfd25ff73a84 None admin] RBAC: Authorizing `identity:get_project(project_id=139a085a71c14dee9992a865eefb6a88)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:29.934065 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5d9171-c91c-4f3f-a212-cfd25ff73a84 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:29.944612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 65/133] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:18:29 2022] GET /identity/v3/projects/139a085a71c14dee9992a865eefb6a88 => generated 312 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:29.949122 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d9171-c91c-4f3f-a212-cfd25ff73a84 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:29.968632 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:30.021737 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7299d6a5-d6f5-4683-8360-3010006e6b9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:30.023561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7299d6a5-d6f5-4683-8360-3010006e6b9c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:30.133499 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5d9171-c91c-4f3f-a212-cfd25ff73a84 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:30.172049 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b7b13-87c2-4bde-b4a3-ffd6e8d867f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:30.173019 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b7b13-87c2-4bde-b4a3-ffd6e8d867f6 None admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:30.173356 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b7b13-87c2-4bde-b4a3-ffd6e8d867f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:30.173644 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b7b13-87c2-4bde-b4a3-ffd6e8d867f6 None admin] PATH_INFO: `/v3/projects/139a085a71c14dee9992a865eefb6a88/users/66b6b05f6db5476b801ab332f119cbdd/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:30.174333 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 69/134] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:18:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:18:30.175817 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03e7d1-deaa-4578-92b2-fb725b71bf02 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:30.175931 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b7b13-87c2-4bde-b4a3-ffd6e8d867f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=139a085a71c14dee9992a865eefb6a88, user_id=66b6b05f6db5476b801ab332f119cbdd, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:30.176124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03e7d1-deaa-4578-92b2-fb725b71bf02 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:30.176437 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a03e7d1-deaa-4578-92b2-fb725b71bf02 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:30.177430 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b7b13-87c2-4bde-b4a3-ffd6e8d867f6 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:30.177779 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 70/135] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:30.181038 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03e7d1-deaa-4578-92b2-fb725b71bf02 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:30.200581 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:30.203820 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 66/136] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 12:18:29 2022] PUT /identity/v3/projects/139a085a71c14dee9992a865eefb6a88/users/66b6b05f6db5476b801ab332f119cbdd/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:18:30.208162 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ea440-a99f-4ca6-bf75-304ec582a178 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:30.208463 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ea440-a99f-4ca6-bf75-304ec582a178 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:30.208747 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307ea440-a99f-4ca6-bf75-304ec582a178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:30.245365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-307ea440-a99f-4ca6-bf75-304ec582a178 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:30.246589 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-307ea440-a99f-4ca6-bf75-304ec582a178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:30.344610 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a03e7d1-deaa-4578-92b2-fb725b71bf02 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:30.372622 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:30.378600 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 67/137] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:30.408451 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eeba42-68fd-41bc-a844-4ced063529d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:30.409313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eeba42-68fd-41bc-a844-4ced063529d1 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:30.409610 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eeba42-68fd-41bc-a844-4ced063529d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:30.409895 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eeba42-68fd-41bc-a844-4ced063529d1 None admin] PATH_INFO: `/v3/projects/2bb553c7fe6e4284b56d30c136b7a041` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:30.411117 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eeba42-68fd-41bc-a844-4ced063529d1 None admin] RBAC: Authorizing `identity:get_project(project_id=2bb553c7fe6e4284b56d30c136b7a041)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:30.412617 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eeba42-68fd-41bc-a844-4ced063529d1 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:30.415137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 71/138] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:18:30 2022] GET /identity/v3/projects/2bb553c7fe6e4284b56d30c136b7a041 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:18:30.417161 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ea440-a99f-4ca6-bf75-304ec582a178 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:30.421028 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eeba42-68fd-41bc-a844-4ced063529d1 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:30.512073 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:30.535232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eeba42-68fd-41bc-a844-4ced063529d1 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:30.576834 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57915f50-cdcf-4385-a4af-c48a4c176686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:30.577998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57915f50-cdcf-4385-a4af-c48a4c176686 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:30.578304 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57915f50-cdcf-4385-a4af-c48a4c176686 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:30.578592 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57915f50-cdcf-4385-a4af-c48a4c176686 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:30.580512 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57915f50-cdcf-4385-a4af-c48a4c176686 None admin] RBAC: Adding query filter params (role.id=c2f1748f5cc24b8181c02f29cfea7aaf, scope.project.id=2bb553c7fe6e4284b56d30c136b7a041, user.id=1f94383c11514c06966b43e65c85d8f3) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:30.580704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307ea440-a99f-4ca6-bf75-304ec582a178 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:30.580875 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57915f50-cdcf-4385-a4af-c48a4c176686 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:30.582881 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57915f50-cdcf-4385-a4af-c48a4c176686 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:30.594077 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 72/139] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:18:30 2022] GET /identity/v3/role_assignments?user.id=1f94383c11514c06966b43e65c85d8f3&scope.project.id=2bb553c7fe6e4284b56d30c136b7a041&role.id=c2f1748f5cc24b8181c02f29cfea7aaf => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:18:30.634423 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc69f8-15fe-41a9-b9f8-246a9d788c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:30.635269 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc69f8-15fe-41a9-b9f8-246a9d788c3e None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:30.635558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc69f8-15fe-41a9-b9f8-246a9d788c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:30.635840 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc69f8-15fe-41a9-b9f8-246a9d788c3e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:30.654655 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc69f8-15fe-41a9-b9f8-246a9d788c3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:30.662866 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-86bc69f8-15fe-41a9-b9f8-246a9d788c3e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 12:18:30.662866 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:30.662866 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:30.662866 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:30.662866 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:30.662866 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:30.662866 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:30.662866 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:30.662866 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:30.662866 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:30.662866 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:30.662866 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:30.662866 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:30.662866 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:30.662866 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:30.662866 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:30.665670 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:30.665670 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:30.665670 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:30.665670 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:30.665670 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:30.665670 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:30.665670 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:30.665670 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:30.665670 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:30.665670 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:30.665670 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:30.665670 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:18:30.665670 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:30.665670 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:30.665670 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:30.665670 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:30.667580 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:30.667580 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:30.667580 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:30.667580 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:30.667580 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:30.667580 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:30.667580 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:30.667580 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:30.667580 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:30.667580 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:30.667580 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 12:18:30.667580 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:18:30.669373 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 68/140] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:18:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:30.677357 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57915f50-cdcf-4385-a4af-c48a4c176686 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:30.820732 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57915f50-cdcf-4385-a4af-c48a4c176686 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:30.859309 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0940749f-8253-464f-a2ca-a0bafbb9e17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:30.860002 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940749f-8253-464f-a2ca-a0bafbb9e17a None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:30.860330 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940749f-8253-464f-a2ca-a0bafbb9e17a None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:30.860617 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940749f-8253-464f-a2ca-a0bafbb9e17a None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:30.861393 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0940749f-8253-464f-a2ca-a0bafbb9e17a None admin] RBAC: Adding query filter params (name=member) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:30.861709 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0940749f-8253-464f-a2ca-a0bafbb9e17a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:30.863123 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0940749f-8253-464f-a2ca-a0bafbb9e17a None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:30.872995 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 73/141] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 12:18:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:30.878631 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc69f8-15fe-41a9-b9f8-246a9d788c3e None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:30.971283 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:31.024249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc69f8-15fe-41a9-b9f8-246a9d788c3e None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:31.083843 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed780f-d02d-44aa-a961-b1ee68d48db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:31.084803 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed780f-d02d-44aa-a961-b1ee68d48db8 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:31.085092 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed780f-d02d-44aa-a961-b1ee68d48db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:31.085897 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed780f-d02d-44aa-a961-b1ee68d48db8 None admin] PATH_INFO: `/v3/users/66b6b05f6db5476b801ab332f119cbdd` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:31.093306 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ed780f-d02d-44aa-a961-b1ee68d48db8 None admin] RBAC: Authorizing `identity:get_user(user_id=66b6b05f6db5476b801ab332f119cbdd)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:31.094828 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ed780f-d02d-44aa-a961-b1ee68d48db8 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:31.096737 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 69/142] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:30 2022] GET /identity/v3/users/66b6b05f6db5476b801ab332f119cbdd => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:31.103194 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0940749f-8253-464f-a2ca-a0bafbb9e17a None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:31.124882 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:31.255740 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0940749f-8253-464f-a2ca-a0bafbb9e17a None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:31.285773 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7196d1b-92ba-430e-a4ab-18c32abd7bb0 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:31.286122 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7196d1b-92ba-430e-a4ab-18c32abd7bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:31.286343 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7196d1b-92ba-430e-a4ab-18c32abd7bb0 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:31.287695 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 70/143] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:31.293478 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e984de-5483-498d-a409-baa07b76dd20 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:31.293693 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e984de-5483-498d-a409-baa07b76dd20 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:31.294059 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e984de-5483-498d-a409-baa07b76dd20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:31.302327 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb551c-740c-476f-ba41-2ddb131eb1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:31.303064 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb551c-740c-476f-ba41-2ddb131eb1c2 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:31.303357 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb551c-740c-476f-ba41-2ddb131eb1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:31.303688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb551c-740c-476f-ba41-2ddb131eb1c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:31.329161 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-d5e984de-5483-498d-a409-baa07b76dd20 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:31.330435 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-d5e984de-5483-498d-a409-baa07b76dd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:31.450481 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 71/144] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:31.453224 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb551c-740c-476f-ba41-2ddb131eb1c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:31.454700 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb551c-740c-476f-ba41-2ddb131eb1c2 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:31.467136 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:31.491054 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e984de-5483-498d-a409-baa07b76dd20 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:31.582101 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:31.658454 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e984de-5483-498d-a409-baa07b76dd20 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:31.660236 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 74/145] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:18:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:18:31.663178 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb551c-740c-476f-ba41-2ddb131eb1c2 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:31.703359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed4dcf-665c-40dd-9ec2-a13c047bca0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:31.704181 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed4dcf-665c-40dd-9ec2-a13c047bca0a None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:31.704474 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed4dcf-665c-40dd-9ec2-a13c047bca0a None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:31.704751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed4dcf-665c-40dd-9ec2-a13c047bca0a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:31.715558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed4dcf-665c-40dd-9ec2-a13c047bca0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:31.731597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-68ed4dcf-665c-40dd-9ec2-a13c047bca0a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 12:18:31.731597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:31.731597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:31.731597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:31.731597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:31.731597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:31.731597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:31.731597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:31.731597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:31.731597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:31.731597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:31.731597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:31.731597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:31.731597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:31.731597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:31.731597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:31.732519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:31.732519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:31.732519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:31.732519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:31.732519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:31.732519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:31.732519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:31.732519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:31.732519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:31.732519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:31.732519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:31.732519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:18:31.732519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:31.732519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:31.732519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:31.732519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:31.733761 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:31.733761 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:31.733761 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:31.733761 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:31.733761 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:31.733761 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:31.733761 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:31.733761 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:31.733761 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:31.733761 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:31.733761 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 12:18:31.733761 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:18:31.733761 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 72/146] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 12:18:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:18:31.739016 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed4dcf-665c-40dd-9ec2-a13c047bca0a None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:31.824372 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb551c-740c-476f-ba41-2ddb131eb1c2 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:31.837675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:31.891694 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8db2de-0e8f-4768-bb59-b53cd4ca855e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:31.892707 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8db2de-0e8f-4768-bb59-b53cd4ca855e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:31.893000 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8db2de-0e8f-4768-bb59-b53cd4ca855e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:31.893284 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8db2de-0e8f-4768-bb59-b53cd4ca855e None admin] PATH_INFO: `/v3/projects/2bb553c7fe6e4284b56d30c136b7a041` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:31.894515 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8db2de-0e8f-4768-bb59-b53cd4ca855e None admin] RBAC: Authorizing `identity:get_project(project_id=2bb553c7fe6e4284b56d30c136b7a041)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:31.896034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8db2de-0e8f-4768-bb59-b53cd4ca855e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:31.898653 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 75/147] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:18:31 2022] GET /identity/v3/projects/2bb553c7fe6e4284b56d30c136b7a041 => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:31.904762 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8db2de-0e8f-4768-bb59-b53cd4ca855e None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:31.984463 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed4dcf-665c-40dd-9ec2-a13c047bca0a None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:32.024201 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:32.055011 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43acfe-746c-4848-b083-33fe02f413a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:32.055689 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43acfe-746c-4848-b083-33fe02f413a2 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:32.055973 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43acfe-746c-4848-b083-33fe02f413a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:32.056276 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43acfe-746c-4848-b083-33fe02f413a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:32.057139 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd43acfe-746c-4848-b083-33fe02f413a2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:32.057561 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd43acfe-746c-4848-b083-33fe02f413a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:32.059047 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd43acfe-746c-4848-b083-33fe02f413a2 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:32.076007 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 73/148] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:18:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:18:32.076162 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8db2de-0e8f-4768-bb59-b53cd4ca855e None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:32.081828 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43acfe-746c-4848-b083-33fe02f413a2 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:32.141802 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73e5eb-f010-4e8c-a4a9-f4950c800bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:32.142484 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73e5eb-f010-4e8c-a4a9-f4950c800bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:32.142773 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73e5eb-f010-4e8c-a4a9-f4950c800bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:32.143056 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b73e5eb-f010-4e8c-a4a9-f4950c800bd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:32.144016 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b73e5eb-f010-4e8c-a4a9-f4950c800bd4 None admin] RBAC: Adding query filter params (role.id=504a45d998f0464893c90fa5429275a8, scope.project.id=2bb553c7fe6e4284b56d30c136b7a041, user.id=66b6b05f6db5476b801ab332f119cbdd) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:32.144468 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b73e5eb-f010-4e8c-a4a9-f4950c800bd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:32.145928 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b73e5eb-f010-4e8c-a4a9-f4950c800bd4 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:32.158127 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 76/149] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:18:31 2022] GET /identity/v3/role_assignments?user.id=66b6b05f6db5476b801ab332f119cbdd&scope.project.id=2bb553c7fe6e4284b56d30c136b7a041&role.id=504a45d998f0464893c90fa5429275a8 => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:18:32.229463 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43acfe-746c-4848-b083-33fe02f413a2 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:32.266993 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01362b42-4506-466b-ae06-476d8c2f3f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:32.267891 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01362b42-4506-466b-ae06-476d8c2f3f25 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:32.268194 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01362b42-4506-466b-ae06-476d8c2f3f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:32.268483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01362b42-4506-466b-ae06-476d8c2f3f25 None admin] PATH_INFO: `/v3/users/66b6b05f6db5476b801ab332f119cbdd` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:32.269795 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01362b42-4506-466b-ae06-476d8c2f3f25 None admin] RBAC: Authorizing `identity:get_user(user_id=66b6b05f6db5476b801ab332f119cbdd)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:32.271307 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01362b42-4506-466b-ae06-476d8c2f3f25 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:32.273566 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 74/150] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:32 2022] GET /identity/v3/users/66b6b05f6db5476b801ab332f119cbdd => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:32.278763 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73e5eb-f010-4e8c-a4a9-f4950c800bd4 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:32.347342 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:32.430611 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b73e5eb-f010-4e8c-a4a9-f4950c800bd4 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:32.485021 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a83ce-5863-4658-8fa1-429d7ab96c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:32.486106 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a83ce-5863-4658-8fa1-429d7ab96c0f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:32.486639 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a83ce-5863-4658-8fa1-429d7ab96c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:32.487110 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a83ce-5863-4658-8fa1-429d7ab96c0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:32.568720 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a83ce-5863-4658-8fa1-429d7ab96c0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:32.571116 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a83ce-5863-4658-8fa1-429d7ab96c0f None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:32.617650 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:32.695409 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 77/151] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:18:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:18:32.700977 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01362b42-4506-466b-ae06-476d8c2f3f25 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:32.766688 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:32.826095 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01362b42-4506-466b-ae06-476d8c2f3f25 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:32.865203 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8cb849-ccaf-4a20-92a9-bb034732abfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:32.866081 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cb849-ccaf-4a20-92a9-bb034732abfe None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:32.866395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cb849-ccaf-4a20-92a9-bb034732abfe None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:32.866750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cb849-ccaf-4a20-92a9-bb034732abfe None admin] PATH_INFO: `/v3/projects/2bb553c7fe6e4284b56d30c136b7a041` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:32.868358 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8cb849-ccaf-4a20-92a9-bb034732abfe None admin] RBAC: Authorizing `identity:get_project(project_id=2bb553c7fe6e4284b56d30c136b7a041)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:32.869810 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8cb849-ccaf-4a20-92a9-bb034732abfe None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:32.871493 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 75/152] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:18:32 2022] GET /identity/v3/projects/2bb553c7fe6e4284b56d30c136b7a041 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:32.877332 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a83ce-5863-4658-8fa1-429d7ab96c0f None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:33.011665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a83ce-5863-4658-8fa1-429d7ab96c0f None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:33.063493 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ffc782-be59-4176-9efa-c757fb1f4947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:33.064663 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ffc782-be59-4176-9efa-c757fb1f4947 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:33.065163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ffc782-be59-4176-9efa-c757fb1f4947 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:33.065633 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ffc782-be59-4176-9efa-c757fb1f4947 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:33.066909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ffc782-be59-4176-9efa-c757fb1f4947 None admin] RBAC: Adding query filter params (role.id=2d4732dacad6444e86d7fee3514afefb, scope.project.id=2bb553c7fe6e4284b56d30c136b7a041, user.id=66b6b05f6db5476b801ab332f119cbdd) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:33.067433 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ffc782-be59-4176-9efa-c757fb1f4947 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:33.069086 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ffc782-be59-4176-9efa-c757fb1f4947 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:33.076311 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 78/153] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:18:32 2022] GET /identity/v3/role_assignments?user.id=66b6b05f6db5476b801ab332f119cbdd&scope.project.id=2bb553c7fe6e4284b56d30c136b7a041&role.id=2d4732dacad6444e86d7fee3514afefb => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:33.206176 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550b4e6-1900-4ca8-ac6c-e682826f01e1 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:33.206506 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550b4e6-1900-4ca8-ac6c-e682826f01e1 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:33.206745 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550b4e6-1900-4ca8-ac6c-e682826f01e1 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:33.208457 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 76/154] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:33.214820 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a8c83-28ea-4027-9c6b-4c8511d469ea None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:33.215455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a8c83-28ea-4027-9c6b-4c8511d469ea None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:33.215934 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a8c83-28ea-4027-9c6b-4c8511d469ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:33.226267 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:33.242954 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-bb1a8c83-28ea-4027-9c6b-4c8511d469ea None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:33.244221 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-bb1a8c83-28ea-4027-9c6b-4c8511d469ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:33.346974 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 79/155] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:33.364561 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2fe313-bd8b-4cad-9ff5-9dad8d80af47 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:33.364561 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2fe313-bd8b-4cad-9ff5-9dad8d80af47 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:33.364561 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2fe313-bd8b-4cad-9ff5-9dad8d80af47 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:33.367068 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 77/156] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:33.372484 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ea646-1e45-4c5a-b226-212dd99c4c5c None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:33.373021 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ea646-1e45-4c5a-b226-212dd99c4c5c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:33.373135 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2fe313-bd8b-4cad-9ff5-9dad8d80af47 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:33.373788 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ea646-1e45-4c5a-b226-212dd99c4c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:33.398317 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-b68ea646-1e45-4c5a-b226-212dd99c4c5c None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:33.400485 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b68ea646-1e45-4c5a-b226-212dd99c4c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:33.498757 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 80/157] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 12:18:33.500417 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0562f-c809-4ffe-9436-95f7490d0097 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:33.500714 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0562f-c809-4ffe-9436-95f7490d0097 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:33.500997 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb0562f-c809-4ffe-9436-95f7490d0097 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:33.503056 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 81/158] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:33.508860 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9bf59-2957-46c5-946c-a06ba933d0a0 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:33.509149 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9bf59-2957-46c5-946c-a06ba933d0a0 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:33.509605 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9bf59-2957-46c5-946c-a06ba933d0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:33.514224 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2fe313-bd8b-4cad-9ff5-9dad8d80af47 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:33.532205 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-bce9bf59-2957-46c5-946c-a06ba933d0a0 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:33.533599 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-bce9bf59-2957-46c5-946c-a06ba933d0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:33.540935 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:33.568225 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4f5ca-1c3d-4857-97a0-dd29d31248c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:33.569149 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4f5ca-1c3d-4857-97a0-dd29d31248c3 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:33.569553 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4f5ca-1c3d-4857-97a0-dd29d31248c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:33.570403 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4f5ca-1c3d-4857-97a0-dd29d31248c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:33.571658 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4f5ca-1c3d-4857-97a0-dd29d31248c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:33.573365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4f5ca-1c3d-4857-97a0-dd29d31248c3 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:33.574986 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 78/159] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:18:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:33.575741 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4f5ca-1c3d-4857-97a0-dd29d31248c3 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:33.605486 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:33.624603 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 82/160] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:33.624603 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce9bf59-2957-46c5-946c-a06ba933d0a0 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:33.688512 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4f5ca-1c3d-4857-97a0-dd29d31248c3 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:33.724406 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cba537-97fe-4410-952c-a932f55c60a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:33.725193 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cba537-97fe-4410-952c-a932f55c60a1 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:33.725477 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cba537-97fe-4410-952c-a932f55c60a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:33.725754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cba537-97fe-4410-952c-a932f55c60a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:33.726062 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce9bf59-2957-46c5-946c-a06ba933d0a0 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:33.737075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cba537-97fe-4410-952c-a932f55c60a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:33.745110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-54cba537-97fe-4410-952c-a932f55c60a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:18:33.745110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:33.745110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:33.745110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:33.745110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:33.745110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:33.745110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:33.745110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:33.745110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:33.745110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:33.745110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:33.745110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:33.745110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:33.745110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:33.745110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:33.745110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:33.746020 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:33.746020 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:33.746020 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:33.746020 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:33.746020 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:33.746020 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:33.746020 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:33.746020 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:33.746020 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:33.746020 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:33.746020 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:33.746020 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:18:33.746020 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:33.746020 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:33.746020 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:33.746020 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:33.746796 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:33.746796 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:33.746796 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:33.746796 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:33.746796 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:33.746796 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:33.746796 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:33.746796 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:33.746796 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:33.746796 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:33.746796 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:18:33.746796 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:18:33.746796 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 79/161] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:18:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:33.747517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cba537-97fe-4410-952c-a932f55c60a1 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:33.770122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a76f0a-8950-46fa-b7dd-0bd25f3c5143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:33.771049 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a76f0a-8950-46fa-b7dd-0bd25f3c5143 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:33.771862 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a76f0a-8950-46fa-b7dd-0bd25f3c5143 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:33.772322 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a76f0a-8950-46fa-b7dd-0bd25f3c5143 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:33.774034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a76f0a-8950-46fa-b7dd-0bd25f3c5143 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:33.776446 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a76f0a-8950-46fa-b7dd-0bd25f3c5143 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:33.796307 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:33.809961 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 83/162] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 12:18:33 2022] POST /identity/v3/users => generated 283 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:18:33.811189 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a76f0a-8950-46fa-b7dd-0bd25f3c5143 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:33.868979 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cba537-97fe-4410-952c-a932f55c60a1 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:33.882973 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:33.917584 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562ebbf-3041-4eb6-9883-6b3b62fce543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:33.918776 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562ebbf-3041-4eb6-9883-6b3b62fce543 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:33.919258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562ebbf-3041-4eb6-9883-6b3b62fce543 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:33.919638 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562ebbf-3041-4eb6-9883-6b3b62fce543 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:33.933330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2562ebbf-3041-4eb6-9883-6b3b62fce543 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:33.942483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-2562ebbf-3041-4eb6-9883-6b3b62fce543 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 12:18:33.942483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:33.942483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:33.942483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:33.942483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:33.942483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:33.942483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:33.942483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:33.942483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:33.942483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:33.942483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:33.942483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:33.942483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:33.942483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:33.942483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:33.942483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:33.943642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:33.943642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:33.943642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:33.943642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:33.943642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:33.943642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:33.943642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:33.943642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:33.943642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:33.943642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:33.943642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:33.943642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:18:33.943642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:33.943642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:33.943642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:33.943642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:33.944574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:33.944574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:33.944574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:33.944574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:33.944574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:33.944574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:33.944574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:33.944574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:33.944574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:33.944574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:33.944574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 12:18:33.944574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:18:33.945181 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 80/163] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:18:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:33.947339 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a76f0a-8950-46fa-b7dd-0bd25f3c5143 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:33.951284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562ebbf-3041-4eb6-9883-6b3b62fce543 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:33.988554 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3202bfa-713f-4541-b812-e27e03c83a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:33.989547 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3202bfa-713f-4541-b812-e27e03c83a4c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:33.990004 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3202bfa-713f-4541-b812-e27e03c83a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:33.990484 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3202bfa-713f-4541-b812-e27e03c83a4c None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:33.991465 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3202bfa-713f-4541-b812-e27e03c83a4c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:33.991951 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3202bfa-713f-4541-b812-e27e03c83a4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:33.993657 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3202bfa-713f-4541-b812-e27e03c83a4c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:34.003228 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 84/164] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:18:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:34.009139 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3202bfa-713f-4541-b812-e27e03c83a4c None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:34.077445 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562ebbf-3041-4eb6-9883-6b3b62fce543 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:34.079786 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:34.123897 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87ec63-d57f-46a9-ada4-64d3158de4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:34.124635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87ec63-d57f-46a9-ada4-64d3158de4b8 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:34.124919 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87ec63-d57f-46a9-ada4-64d3158de4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:34.125222 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87ec63-d57f-46a9-ada4-64d3158de4b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:34.125953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be87ec63-d57f-46a9-ada4-64d3158de4b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:34.126260 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be87ec63-d57f-46a9-ada4-64d3158de4b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:34.127658 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be87ec63-d57f-46a9-ada4-64d3158de4b8 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:34.140285 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 81/165] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 12:18:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:34.141510 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651e796-1fdf-4a52-be2f-f3cd4a268125 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:34.141819 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651e796-1fdf-4a52-be2f-f3cd4a268125 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:34.142069 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651e796-1fdf-4a52-be2f-f3cd4a268125 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:34.144289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 82/166] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:34.145466 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651e796-1fdf-4a52-be2f-f3cd4a268125 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:34.145582 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3202bfa-713f-4541-b812-e27e03c83a4c None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:34.160037 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:34.197421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df9f72-e96b-4652-a59e-4a7293c8b603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:34.198953 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df9f72-e96b-4652-a59e-4a7293c8b603 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:34.199246 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df9f72-e96b-4652-a59e-4a7293c8b603 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:34.199528 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df9f72-e96b-4652-a59e-4a7293c8b603 None admin] PATH_INFO: `/v3/users/89b57772ff384ce6af49426c38787139` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:34.201498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5df9f72-e96b-4652-a59e-4a7293c8b603 None admin] RBAC: Authorizing `identity:get_user(user_id=89b57772ff384ce6af49426c38787139)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:34.203266 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5df9f72-e96b-4652-a59e-4a7293c8b603 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:34.205320 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 85/167] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:34 2022] GET /identity/v3/users/89b57772ff384ce6af49426c38787139 => generated 247 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:18:34.206833 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c0b06-c2c6-457a-bdd8-a550a8cd0e12 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:34.207149 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c0b06-c2c6-457a-bdd8-a550a8cd0e12 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:34.207433 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c0b06-c2c6-457a-bdd8-a550a8cd0e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:34.225623 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-1f8c0b06-c2c6-457a-bdd8-a550a8cd0e12 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:34.227558 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1f8c0b06-c2c6-457a-bdd8-a550a8cd0e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:34.277465 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651e796-1fdf-4a52-be2f-f3cd4a268125 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:34.315046 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 86/168] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:34.316009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c0b06-c2c6-457a-bdd8-a550a8cd0e12 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:34.323343 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62735e50-3a94-4d6a-87f9-1f852e0e4087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:34.324255 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62735e50-3a94-4d6a-87f9-1f852e0e4087 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:34.324545 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62735e50-3a94-4d6a-87f9-1f852e0e4087 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:34.324822 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62735e50-3a94-4d6a-87f9-1f852e0e4087 None admin] PATH_INFO: `/v3/users/66b6b05f6db5476b801ab332f119cbdd` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:34.326025 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62735e50-3a94-4d6a-87f9-1f852e0e4087 None admin] RBAC: Authorizing `identity:get_user(user_id=66b6b05f6db5476b801ab332f119cbdd)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:34.327543 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62735e50-3a94-4d6a-87f9-1f852e0e4087 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:34.329394 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 83/169] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:34 2022] GET /identity/v3/users/66b6b05f6db5476b801ab332f119cbdd => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:34.334642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62735e50-3a94-4d6a-87f9-1f852e0e4087 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:34.380392 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:34.458575 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c0b06-c2c6-457a-bdd8-a550a8cd0e12 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:34.464321 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62735e50-3a94-4d6a-87f9-1f852e0e4087 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:34.469539 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:34.505029 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda92186-b9f8-44d6-99d7-29d1a168d918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:34.505852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda92186-b9f8-44d6-99d7-29d1a168d918 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:34.506144 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda92186-b9f8-44d6-99d7-29d1a168d918 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:34.506423 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda92186-b9f8-44d6-99d7-29d1a168d918 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:34.512274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b0763-cec8-4607-bf5a-b0ff5f99b247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:34.512945 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b0763-cec8-4607-bf5a-b0ff5f99b247 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:34.513236 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b0763-cec8-4607-bf5a-b0ff5f99b247 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:34.513524 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b0763-cec8-4607-bf5a-b0ff5f99b247 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:34.585718 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda92186-b9f8-44d6-99d7-29d1a168d918 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:34.587393 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda92186-b9f8-44d6-99d7-29d1a168d918 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:34.613443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2b0763-cec8-4607-bf5a-b0ff5f99b247 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:34.615487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2b0763-cec8-4607-bf5a-b0ff5f99b247 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:34.652282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:34.736086 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 87/170] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:18:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:18:34.740323 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda92186-b9f8-44d6-99d7-29d1a168d918 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:34.760412 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:34.802026 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 84/171] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:18:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:18:34.803605 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b0763-cec8-4607-bf5a-b0ff5f99b247 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:34.868373 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda92186-b9f8-44d6-99d7-29d1a168d918 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:34.903763 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928fb91-7386-43b5-9f15-771e724dd2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:34.904682 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928fb91-7386-43b5-9f15-771e724dd2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:34.904969 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928fb91-7386-43b5-9f15-771e724dd2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:34.905305 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928fb91-7386-43b5-9f15-771e724dd2e5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:34.922208 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2928fb91-7386-43b5-9f15-771e724dd2e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:34.932217 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-2928fb91-7386-43b5-9f15-771e724dd2e5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 12:18:34.932217 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:34.932217 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:34.932217 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:34.932217 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:34.932217 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:34.932217 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:34.932217 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:34.932217 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:34.932217 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:34.932217 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:34.932217 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:34.932217 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:34.932217 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:34.932217 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:34.932217 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:34.933101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:34.933101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:34.933101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:34.933101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:34.933101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:34.933101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:34.933101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:34.933101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:34.933101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:34.933101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:34.933101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:34.933101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:18:34.933101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:34.933101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:34.933101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:34.933101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:34.933960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:34.933960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:34.933960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:34.933960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:34.933960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:34.933960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:34.933960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:34.933960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:34.933960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:34.933960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:34.933960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 12:18:34.933960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:18:34.938367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 88/172] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:18:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:34.939226 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928fb91-7386-43b5-9f15-771e724dd2e5 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:35.021058 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:35.057140 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928fb91-7386-43b5-9f15-771e724dd2e5 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:35.062968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b0763-cec8-4607-bf5a-b0ff5f99b247 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:35.094173 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a14855-983a-4c8a-9207-5a86a1c4fb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:35.095114 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a14855-983a-4c8a-9207-5a86a1c4fb85 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:35.095400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a14855-983a-4c8a-9207-5a86a1c4fb85 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:35.095734 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a14855-983a-4c8a-9207-5a86a1c4fb85 None admin] PATH_INFO: `/v3/projects/139a085a71c14dee9992a865eefb6a88` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:35.096966 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a14855-983a-4c8a-9207-5a86a1c4fb85 None admin] RBAC: Authorizing `identity:get_project(project_id=139a085a71c14dee9992a865eefb6a88)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:35.098546 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a14855-983a-4c8a-9207-5a86a1c4fb85 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:35.100268 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 89/173] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:18:34 2022] GET /identity/v3/projects/139a085a71c14dee9992a865eefb6a88 => generated 312 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:35.101692 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a14855-983a-4c8a-9207-5a86a1c4fb85 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:35.160980 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e8f71-6bba-431a-b367-724d83292aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:35.161793 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e8f71-6bba-431a-b367-724d83292aed None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:35.162084 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e8f71-6bba-431a-b367-724d83292aed None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:35.162366 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e8f71-6bba-431a-b367-724d83292aed None admin] PATH_INFO: `/v3/projects/2bb553c7fe6e4284b56d30c136b7a041` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:35.164136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e8f71-6bba-431a-b367-724d83292aed None admin] RBAC: Authorizing `identity:get_project(project_id=2bb553c7fe6e4284b56d30c136b7a041)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:35.165644 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e8f71-6bba-431a-b367-724d83292aed None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:35.167382 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 85/174] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:18:34 2022] GET /identity/v3/projects/2bb553c7fe6e4284b56d30c136b7a041 => generated 298 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:35.168159 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e8f71-6bba-431a-b367-724d83292aed None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:35.178944 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:35.218700 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a14855-983a-4c8a-9207-5a86a1c4fb85 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:35.256667 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc33d9-6914-4dc1-a140-4c4c1ea98d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:35.257639 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc33d9-6914-4dc1-a140-4c4c1ea98d83 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:35.257936 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc33d9-6914-4dc1-a140-4c4c1ea98d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:35.258364 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc33d9-6914-4dc1-a140-4c4c1ea98d83 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:35.259241 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc33d9-6914-4dc1-a140-4c4c1ea98d83 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:35.259561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc33d9-6914-4dc1-a140-4c4c1ea98d83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:35.260995 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc33d9-6914-4dc1-a140-4c4c1ea98d83 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:35.263732 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:35.279401 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 90/175] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 12:18:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:35.285607 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc33d9-6914-4dc1-a140-4c4c1ea98d83 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:35.417256 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc33d9-6914-4dc1-a140-4c4c1ea98d83 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:35.454054 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964b215c-c877-45c6-b369-d89c4130d982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:35.454726 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b215c-c877-45c6-b369-d89c4130d982 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:35.455014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b215c-c877-45c6-b369-d89c4130d982 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:35.455297 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b215c-c877-45c6-b369-d89c4130d982 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:35.456137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964b215c-c877-45c6-b369-d89c4130d982 None admin] RBAC: Adding query filter params (role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8, scope.project.id=2bb553c7fe6e4284b56d30c136b7a041, user.id=89b57772ff384ce6af49426c38787139) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:35.456484 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964b215c-c877-45c6-b369-d89c4130d982 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:35.458097 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964b215c-c877-45c6-b369-d89c4130d982 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:35.458284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e8f71-6bba-431a-b367-724d83292aed None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:35.465922 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 91/176] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:18:35 2022] GET /identity/v3/role_assignments?user.id=89b57772ff384ce6af49426c38787139&scope.project.id=2bb553c7fe6e4284b56d30c136b7a041&role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:35.466738 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964b215c-c877-45c6-b369-d89c4130d982 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:35.535541 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:35.568914 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab2f6f-7a04-4134-9d64-dd51f1ca4391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:35.570160 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab2f6f-7a04-4134-9d64-dd51f1ca4391 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:35.570456 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab2f6f-7a04-4134-9d64-dd51f1ca4391 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:35.570740 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab2f6f-7a04-4134-9d64-dd51f1ca4391 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:35.571599 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ab2f6f-7a04-4134-9d64-dd51f1ca4391 None admin] RBAC: Adding query filter params (role.id=504a45d998f0464893c90fa5429275a8, scope.project.id=139a085a71c14dee9992a865eefb6a88, user.id=66b6b05f6db5476b801ab332f119cbdd) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:35.571917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ab2f6f-7a04-4134-9d64-dd51f1ca4391 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:35.573446 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ab2f6f-7a04-4134-9d64-dd51f1ca4391 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:35.590674 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 86/177] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:18:35 2022] GET /identity/v3/role_assignments?user.id=66b6b05f6db5476b801ab332f119cbdd&scope.project.id=139a085a71c14dee9992a865eefb6a88&role.id=504a45d998f0464893c90fa5429275a8 => generated 608 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:35.592957 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964b215c-c877-45c6-b369-d89c4130d982 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:35.637823 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dde8c33-ec3d-40b5-9f2c-2654c8cef059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:35.638816 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde8c33-ec3d-40b5-9f2c-2654c8cef059 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:35.639105 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde8c33-ec3d-40b5-9f2c-2654c8cef059 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:35.639435 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde8c33-ec3d-40b5-9f2c-2654c8cef059 None admin] PATH_INFO: `/v3/users/1f94383c11514c06966b43e65c85d8f3` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:35.640827 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dde8c33-ec3d-40b5-9f2c-2654c8cef059 None admin] RBAC: Authorizing `identity:get_user(user_id=1f94383c11514c06966b43e65c85d8f3)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:35.642339 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dde8c33-ec3d-40b5-9f2c-2654c8cef059 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:35.644161 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 92/178] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:35 2022] GET /identity/v3/users/1f94383c11514c06966b43e65c85d8f3 => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:35.655419 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab2f6f-7a04-4134-9d64-dd51f1ca4391 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:35.757136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:35.807376 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab2f6f-7a04-4134-9d64-dd51f1ca4391 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:35.857010 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0932d98-bada-4b85-9422-6ecb9ebf5c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:35.857903 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0932d98-bada-4b85-9422-6ecb9ebf5c69 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:35.858184 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0932d98-bada-4b85-9422-6ecb9ebf5c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:35.858462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0932d98-bada-4b85-9422-6ecb9ebf5c69 None admin] PATH_INFO: `/v3/projects/2bb553c7fe6e4284b56d30c136b7a041` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:35.859777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0932d98-bada-4b85-9422-6ecb9ebf5c69 None admin] RBAC: Authorizing `identity:get_project(project_id=2bb553c7fe6e4284b56d30c136b7a041)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:35.861258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0932d98-bada-4b85-9422-6ecb9ebf5c69 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:35.862944 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 87/179] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:18:35 2022] GET /identity/v3/projects/2bb553c7fe6e4284b56d30c136b7a041 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:35.869613 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dde8c33-ec3d-40b5-9f2c-2654c8cef059 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:36.016189 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dde8c33-ec3d-40b5-9f2c-2654c8cef059 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:36.050290 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:36.057733 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836eb6a-e85d-49d2-8b28-bf175b68e220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:36.058638 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836eb6a-e85d-49d2-8b28-bf175b68e220 None admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:36.058927 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836eb6a-e85d-49d2-8b28-bf175b68e220 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:36.059208 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836eb6a-e85d-49d2-8b28-bf175b68e220 None admin] PATH_INFO: `/v3/projects/2bb553c7fe6e4284b56d30c136b7a041/users/1f94383c11514c06966b43e65c85d8f3/roles/c2f1748f5cc24b8181c02f29cfea7aaf` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:36.061252 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8836eb6a-e85d-49d2-8b28-bf175b68e220 None admin] RBAC: Authorizing `identity:create_grant(project_id=2bb553c7fe6e4284b56d30c136b7a041, user_id=1f94383c11514c06966b43e65c85d8f3, role_id=c2f1748f5cc24b8181c02f29cfea7aaf)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:36.062715 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8836eb6a-e85d-49d2-8b28-bf175b68e220 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:36.078499 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 93/180] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 12:18:35 2022] PUT /identity/v3/projects/2bb553c7fe6e4284b56d30c136b7a041/users/1f94383c11514c06966b43e65c85d8f3/roles/c2f1748f5cc24b8181c02f29cfea7aaf => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:18:36.703225 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5aa67-9c8c-4de8-b6ef-6603410bc8bf None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:36.703599 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5aa67-9c8c-4de8-b6ef-6603410bc8bf None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:36.703809 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5aa67-9c8c-4de8-b6ef-6603410bc8bf None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:36.705182 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 88/181] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:36.710372 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b93f9-c6b3-4822-9a6b-247c9df657e8 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:36.710665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b93f9-c6b3-4822-9a6b-247c9df657e8 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:36.711012 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b93f9-c6b3-4822-9a6b-247c9df657e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:36.728416 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-8d5b93f9-c6b3-4822-9a6b-247c9df657e8 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:36.729587 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-8d5b93f9-c6b3-4822-9a6b-247c9df657e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:36.802854 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 94/182] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:36.826895 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5aa67-9c8c-4de8-b6ef-6603410bc8bf None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:36.861078 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51998a7b-431b-4d25-9b9f-e62ca981c3c8 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:36.861719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51998a7b-431b-4d25-9b9f-e62ca981c3c8 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:36.862198 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51998a7b-431b-4d25-9b9f-e62ca981c3c8 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:36.863789 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 95/183] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:18:36.869265 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98333ddf-f5fa-450f-bb66-8cca864bf910 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:36.869830 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98333ddf-f5fa-450f-bb66-8cca864bf910 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:36.870757 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98333ddf-f5fa-450f-bb66-8cca864bf910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:36.888654 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-98333ddf-f5fa-450f-bb66-8cca864bf910 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:36.890166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-98333ddf-f5fa-450f-bb66-8cca864bf910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:36.942162 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5aa67-9c8c-4de8-b6ef-6603410bc8bf None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:36.980021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e48d0-f595-41e3-bab0-bfb18aa83e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:36.980392 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 96/184] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 12:18:36.985316 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e48d0-f595-41e3-bab0-bfb18aa83e2c None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:36.985423 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e48d0-f595-41e3-bab0-bfb18aa83e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:36.985748 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e48d0-f595-41e3-bab0-bfb18aa83e2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:36.987620 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:36.998607 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e48d0-f595-41e3-bab0-bfb18aa83e2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:37.005443 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98333ddf-f5fa-450f-bb66-8cca864bf910 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:37.006443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-380e48d0-f595-41e3-bab0-bfb18aa83e2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:18:37.006443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:37.006443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:37.006443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:37.006443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:37.006443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:37.006443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:37.006443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:37.006443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:37.006443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:37.006443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:37.006443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:37.006443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:37.006443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:37.006443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:37.006443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:37.007887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:37.007887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:37.007887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:37.007887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:37.007887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:37.007887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:37.007887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:37.007887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:37.007887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:37.007887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:37.007887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:37.007887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:18:37.007887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:37.007887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:37.007887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:37.007887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:37.009044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:37.009044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:37.009044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:37.009044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:37.009044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:37.009044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:37.009044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:37.009044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:37.009044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:37.009044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:37.009044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:18:37.009044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:18:37.009044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 89/185] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:18:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:37.010271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e5c9c7-7faa-4235-b94b-a2fc077525d4 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:37.010564 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e5c9c7-7faa-4235-b94b-a2fc077525d4 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:37.010851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e5c9c7-7faa-4235-b94b-a2fc077525d4 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:37.012157 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 90/186] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:37.014208 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e5c9c7-7faa-4235-b94b-a2fc077525d4 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:37.064308 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:37.115575 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98333ddf-f5fa-450f-bb66-8cca864bf910 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:37.125904 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e5c9c7-7faa-4235-b94b-a2fc077525d4 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:37.155717 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e254560-37d3-4fc9-b976-c0eb6da96bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:37.156941 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e254560-37d3-4fc9-b976-c0eb6da96bbe None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:37.157321 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e254560-37d3-4fc9-b976-c0eb6da96bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:37.157631 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e254560-37d3-4fc9-b976-c0eb6da96bbe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:37.162403 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd49931e-23cd-414f-b6c5-f3d90f0fdb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:37.163093 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49931e-23cd-414f-b6c5-f3d90f0fdb05 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:37.164012 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49931e-23cd-414f-b6c5-f3d90f0fdb05 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:37.164400 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49931e-23cd-414f-b6c5-f3d90f0fdb05 None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:37.165150 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd49931e-23cd-414f-b6c5-f3d90f0fdb05 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:37.165469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd49931e-23cd-414f-b6c5-f3d90f0fdb05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:37.166859 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd49931e-23cd-414f-b6c5-f3d90f0fdb05 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:37.170411 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e254560-37d3-4fc9-b976-c0eb6da96bbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:37.175328 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 91/187] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:18:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:37.176559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eeb096-bff3-42a1-8480-4db6f7796f68 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:37.176850 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eeb096-bff3-42a1-8480-4db6f7796f68 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:37.177131 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eeb096-bff3-42a1-8480-4db6f7796f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:37.179872 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-7e254560-37d3-4fc9-b976-c0eb6da96bbe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 12:18:37.179872 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:37.179872 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:37.179872 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:37.179872 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:37.179872 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:37.179872 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:37.179872 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:37.179872 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:37.179872 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:37.179872 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:37.179872 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:37.179872 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:37.179872 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:37.179872 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:37.179872 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:37.180906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:37.180906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:37.180906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:37.180906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:37.180906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:37.180906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:37.180906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:37.180906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:37.180906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:37.180906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:37.180906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:37.180906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:18:37.180906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:37.180906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:37.180906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:37.180906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:37.181715 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:37.181715 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:37.181715 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:37.181715 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:37.181715 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:37.181715 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:37.181715 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:37.181715 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:37.181715 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:37.181715 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:37.181715 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 12:18:37.181715 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:18:37.183985 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 97/188] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 12:18:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:18:37.184995 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e254560-37d3-4fc9-b976-c0eb6da96bbe None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:37.200107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-21eeb096-bff3-42a1-8480-4db6f7796f68 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:37.201401 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-21eeb096-bff3-42a1-8480-4db6f7796f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:37.258427 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:37.284341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 92/189] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:37.284341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eeb096-bff3-42a1-8480-4db6f7796f68 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:37.307802 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e254560-37d3-4fc9-b976-c0eb6da96bbe None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:37.318942 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:37.344687 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d31412-99c1-44a1-aa85-bf83c6454e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:37.345576 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d31412-99c1-44a1-aa85-bf83c6454e11 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:37.345868 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d31412-99c1-44a1-aa85-bf83c6454e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:37.346148 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d31412-99c1-44a1-aa85-bf83c6454e11 None admin] PATH_INFO: `/v3/users/f17a529958ca41cab7529a7d68f90ee8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:37.362467 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d31412-99c1-44a1-aa85-bf83c6454e11 None admin] RBAC: Authorizing `identity:get_user(user_id=f17a529958ca41cab7529a7d68f90ee8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:37.364028 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d31412-99c1-44a1-aa85-bf83c6454e11 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:37.365777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 98/190] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:37 2022] GET /identity/v3/users/f17a529958ca41cab7529a7d68f90ee8 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:37.371826 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d31412-99c1-44a1-aa85-bf83c6454e11 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:37.401513 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eeb096-bff3-42a1-8480-4db6f7796f68 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:37.442801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c9504-cac7-452d-91da-3c8eca6111c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:37.443469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c9504-cac7-452d-91da-3c8eca6111c3 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:37.443759 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c9504-cac7-452d-91da-3c8eca6111c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:37.444044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c9504-cac7-452d-91da-3c8eca6111c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:37.445586 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1c9504-cac7-452d-91da-3c8eca6111c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:37.445998 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1c9504-cac7-452d-91da-3c8eca6111c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:37.447398 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1c9504-cac7-452d-91da-3c8eca6111c3 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:37.455640 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 93/191] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:18:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:37.456270 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c9504-cac7-452d-91da-3c8eca6111c3 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:37.479351 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d31412-99c1-44a1-aa85-bf83c6454e11 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:37.509604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:37.519030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2eae40-27d5-4117-9c52-bb5c39bea7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:37.519832 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2eae40-27d5-4117-9c52-bb5c39bea7fe None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:37.520121 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2eae40-27d5-4117-9c52-bb5c39bea7fe None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:37.520428 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2eae40-27d5-4117-9c52-bb5c39bea7fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:37.521672 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2eae40-27d5-4117-9c52-bb5c39bea7fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:37.523282 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2eae40-27d5-4117-9c52-bb5c39bea7fe None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:37.524999 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 99/192] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:18:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:37.525634 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2eae40-27d5-4117-9c52-bb5c39bea7fe None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:37.561540 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:37.574881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c9504-cac7-452d-91da-3c8eca6111c3 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:37.610775 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf3430-a08e-4028-97ec-b1cfe32f713d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:37.611995 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf3430-a08e-4028-97ec-b1cfe32f713d None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:37.613014 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf3430-a08e-4028-97ec-b1cfe32f713d None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:37.613757 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf3430-a08e-4028-97ec-b1cfe32f713d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:37.633631 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2eae40-27d5-4117-9c52-bb5c39bea7fe None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:37.670656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d42ac-99f2-4162-a2b6-f04d7b16a14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:37.672186 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d42ac-99f2-4162-a2b6-f04d7b16a14c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:37.672664 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d42ac-99f2-4162-a2b6-f04d7b16a14c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:37.672948 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d42ac-99f2-4162-a2b6-f04d7b16a14c None admin] PATH_INFO: `/v3/users/f17a529958ca41cab7529a7d68f90ee8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:37.674163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d42ac-99f2-4162-a2b6-f04d7b16a14c None admin] RBAC: Authorizing `identity:get_user(user_id=f17a529958ca41cab7529a7d68f90ee8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:37.675597 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d42ac-99f2-4162-a2b6-f04d7b16a14c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:37.682410 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 100/193] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:37 2022] GET /identity/v3/users/f17a529958ca41cab7529a7d68f90ee8 => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:37.683004 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d42ac-99f2-4162-a2b6-f04d7b16a14c None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:37.694519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf3430-a08e-4028-97ec-b1cfe32f713d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:37.696380 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf3430-a08e-4028-97ec-b1cfe32f713d None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:37.748075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:37.795238 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d42ac-99f2-4162-a2b6-f04d7b16a14c None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:37.799781 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:37.803471 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 94/194] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:18:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:18:37.804273 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf3430-a08e-4028-97ec-b1cfe32f713d None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:37.840434 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6874d1-4cf2-45c8-a2ed-f7b27c1036c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:37.841121 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6874d1-4cf2-45c8-a2ed-f7b27c1036c9 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:37.841406 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6874d1-4cf2-45c8-a2ed-f7b27c1036c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:37.841686 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6874d1-4cf2-45c8-a2ed-f7b27c1036c9 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:37.842405 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6874d1-4cf2-45c8-a2ed-f7b27c1036c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:37.843855 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6874d1-4cf2-45c8-a2ed-f7b27c1036c9 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:37.866359 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 101/195] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 12:18:37 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:18:37.873269 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6874d1-4cf2-45c8-a2ed-f7b27c1036c9 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:37.941602 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf3430-a08e-4028-97ec-b1cfe32f713d None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:37.990104 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c23e7-51f6-4411-a20f-414b7aaa1e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:37.990981 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c23e7-51f6-4411-a20f-414b7aaa1e3e None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:37.991951 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c23e7-51f6-4411-a20f-414b7aaa1e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:37.992365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c23e7-51f6-4411-a20f-414b7aaa1e3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:38.015579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:38.017769 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6874d1-4cf2-45c8-a2ed-f7b27c1036c9 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:38.070521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e364a8-421f-44bc-94e2-a8bb1c921233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:38.072137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e364a8-421f-44bc-94e2-a8bb1c921233 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:38.072619 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e364a8-421f-44bc-94e2-a8bb1c921233 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:38.073291 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e364a8-421f-44bc-94e2-a8bb1c921233 None admin] PATH_INFO: `/v3/projects/49c3998144de4918b868b834ea2918dc` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:38.075909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e364a8-421f-44bc-94e2-a8bb1c921233 None admin] RBAC: Authorizing `identity:get_project(project_id=49c3998144de4918b868b834ea2918dc)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:38.077852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e364a8-421f-44bc-94e2-a8bb1c921233 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:38.080389 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 102/196] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:18:37 2022] GET /identity/v3/projects/49c3998144de4918b868b834ea2918dc => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:38.086311 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e364a8-421f-44bc-94e2-a8bb1c921233 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:38.088294 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58c23e7-51f6-4411-a20f-414b7aaa1e3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:38.089796 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58c23e7-51f6-4411-a20f-414b7aaa1e3e None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:38.122901 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:38.214688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e364a8-421f-44bc-94e2-a8bb1c921233 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:38.231538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 95/197] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:18:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:18:38.237430 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c23e7-51f6-4411-a20f-414b7aaa1e3e None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:38.258634 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df684f-cc41-4d48-ac76-d3d8b039a6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:38.259638 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df684f-cc41-4d48-ac76-d3d8b039a6fc None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:38.260134 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df684f-cc41-4d48-ac76-d3d8b039a6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:38.260540 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df684f-cc41-4d48-ac76-d3d8b039a6fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:38.261532 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df684f-cc41-4d48-ac76-d3d8b039a6fc None admin] RBAC: Adding query filter params (role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8, scope.project.id=49c3998144de4918b868b834ea2918dc, user.id=f17a529958ca41cab7529a7d68f90ee8) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:38.261920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df684f-cc41-4d48-ac76-d3d8b039a6fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:38.263479 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df684f-cc41-4d48-ac76-d3d8b039a6fc None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:38.270676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 103/198] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:18:38 2022] GET /identity/v3/role_assignments?user.id=f17a529958ca41cab7529a7d68f90ee8&scope.project.id=49c3998144de4918b868b834ea2918dc&role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:38.300594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:38.369441 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c23e7-51f6-4411-a20f-414b7aaa1e3e None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:38.400835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963ec9c-9e75-4865-aa8a-c7704d1ea782 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:38.401456 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963ec9c-9e75-4865-aa8a-c7704d1ea782 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:38.401928 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7963ec9c-9e75-4865-aa8a-c7704d1ea782 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:38.403463 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 104/199] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:38.410089 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eee4a4-2a41-4a61-9910-2cd2e1e77bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:38.411038 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eee4a4-2a41-4a61-9910-2cd2e1e77bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:38.411331 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eee4a4-2a41-4a61-9910-2cd2e1e77bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:38.411568 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784fa03-7e6e-4529-8d21-3c1c7f792c5c None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:38.411688 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eee4a4-2a41-4a61-9910-2cd2e1e77bc8 None admin] PATH_INFO: `/v3/projects/49c3998144de4918b868b834ea2918dc` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:38.412154 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784fa03-7e6e-4529-8d21-3c1c7f792c5c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:38.412678 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784fa03-7e6e-4529-8d21-3c1c7f792c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:38.414298 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eee4a4-2a41-4a61-9910-2cd2e1e77bc8 None admin] RBAC: Authorizing `identity:get_project(project_id=49c3998144de4918b868b834ea2918dc)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:38.416125 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eee4a4-2a41-4a61-9910-2cd2e1e77bc8 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:38.418781 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 96/200] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:18:38 2022] GET /identity/v3/projects/49c3998144de4918b868b834ea2918dc => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:18:38.425370 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eee4a4-2a41-4a61-9910-2cd2e1e77bc8 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:38.434998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-3784fa03-7e6e-4529-8d21-3c1c7f792c5c None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:38.436611 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3784fa03-7e6e-4529-8d21-3c1c7f792c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:38.536199 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 105/201] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 12:18:38.567279 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eee4a4-2a41-4a61-9910-2cd2e1e77bc8 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:38.574910 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784fa03-7e6e-4529-8d21-3c1c7f792c5c None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:38.608571 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:38.633056 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:38.641606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1b59b-5b65-4d52-b937-6816e4401fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:38.642462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1b59b-5b65-4d52-b937-6816e4401fcb None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:38.642592 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1b59b-5b65-4d52-b937-6816e4401fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:38.642815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1b59b-5b65-4d52-b937-6816e4401fcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:38.643661 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f1b59b-5b65-4d52-b937-6816e4401fcb None admin] RBAC: Adding query filter params (role.id=2d4732dacad6444e86d7fee3514afefb, scope.project.id=49c3998144de4918b868b834ea2918dc, user.id=f17a529958ca41cab7529a7d68f90ee8) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:38.643971 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f1b59b-5b65-4d52-b937-6816e4401fcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:38.646193 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f1b59b-5b65-4d52-b937-6816e4401fcb None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:38.659125 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 97/202] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:18:38 2022] GET /identity/v3/role_assignments?user.id=f17a529958ca41cab7529a7d68f90ee8&scope.project.id=49c3998144de4918b868b834ea2918dc&role.id=2d4732dacad6444e86d7fee3514afefb => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:18:38.728494 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784fa03-7e6e-4529-8d21-3c1c7f792c5c None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:38.771869 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6a6e3a-9531-4168-b24c-e8d7dc1168b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:38.773262 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6a6e3a-9531-4168-b24c-e8d7dc1168b9 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:38.773545 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6a6e3a-9531-4168-b24c-e8d7dc1168b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:38.773821 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6a6e3a-9531-4168-b24c-e8d7dc1168b9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:38.793339 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6a6e3a-9531-4168-b24c-e8d7dc1168b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:38.803440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-fd6a6e3a-9531-4168-b24c-e8d7dc1168b9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 12:18:38.803440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:38.803440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:38.803440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:38.803440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:38.803440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:38.803440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:38.803440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:38.803440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:38.803440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:38.803440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:38.803440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:38.803440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:38.803440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:38.803440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:38.803440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:38.807809 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:38.807809 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:38.807809 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:38.807809 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:38.807809 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:38.807809 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:38.807809 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:38.807809 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:38.807809 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:38.807809 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:38.807809 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:38.807809 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:18:38.807809 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:38.807809 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:38.807809 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:38.807809 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:38.808677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:38.808677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:38.808677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:38.808677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:38.808677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:38.808677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:38.808677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:38.808677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:38.808677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:38.808677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:38.808677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 12:18:38.808677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:18:38.812285 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 106/203] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:18:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:38.817597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1b59b-5b65-4d52-b937-6816e4401fcb None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:38.963225 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1b59b-5b65-4d52-b937-6816e4401fcb None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:38.999289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8fc22b-882d-46ed-9028-3e7e3f58975e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:38.999966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fc22b-882d-46ed-9028-3e7e3f58975e None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:39.000290 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fc22b-882d-46ed-9028-3e7e3f58975e None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:39.000572 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fc22b-882d-46ed-9028-3e7e3f58975e None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:39.001332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8fc22b-882d-46ed-9028-3e7e3f58975e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:39.001645 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8fc22b-882d-46ed-9028-3e7e3f58975e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:39.003045 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8fc22b-882d-46ed-9028-3e7e3f58975e None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:39.017204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 98/204] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 12:18:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:39.025064 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6a6e3a-9531-4168-b24c-e8d7dc1168b9 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:39.164440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:39.189153 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6a6e3a-9531-4168-b24c-e8d7dc1168b9 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:39.245018 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc07d5-4d67-4ab3-a558-5d338c0011b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:39.245831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc07d5-4d67-4ab3-a558-5d338c0011b5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:39.246117 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc07d5-4d67-4ab3-a558-5d338c0011b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:39.246394 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc07d5-4d67-4ab3-a558-5d338c0011b5 None admin] PATH_INFO: `/v3/users/1f94383c11514c06966b43e65c85d8f3` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:39.247651 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc07d5-4d67-4ab3-a558-5d338c0011b5 None admin] RBAC: Authorizing `identity:get_user(user_id=1f94383c11514c06966b43e65c85d8f3)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:39.249141 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc07d5-4d67-4ab3-a558-5d338c0011b5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:39.251160 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 107/205] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:39 2022] GET /identity/v3/users/1f94383c11514c06966b43e65c85d8f3 => generated 289 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:39.256577 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8fc22b-882d-46ed-9028-3e7e3f58975e None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:39.314635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:39.373073 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8fc22b-882d-46ed-9028-3e7e3f58975e None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:39.420651 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b8dc1-0119-4a14-bdbd-701fb536a1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:39.421413 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b8dc1-0119-4a14-bdbd-701fb536a1bc None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:39.421763 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b8dc1-0119-4a14-bdbd-701fb536a1bc None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:39.422095 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b8dc1-0119-4a14-bdbd-701fb536a1bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:39.511695 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b8dc1-0119-4a14-bdbd-701fb536a1bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:39.513197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b8dc1-0119-4a14-bdbd-701fb536a1bc None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:39.563802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:39.636610 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 99/206] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:18:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:18:39.642079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc07d5-4d67-4ab3-a558-5d338c0011b5 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:39.774580 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc07d5-4d67-4ab3-a558-5d338c0011b5 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:39.810230 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a36f88-f5f0-49bd-a513-227bb0a6b2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:39.811436 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a36f88-f5f0-49bd-a513-227bb0a6b2ec None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:39.811942 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a36f88-f5f0-49bd-a513-227bb0a6b2ec None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:39.812452 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a36f88-f5f0-49bd-a513-227bb0a6b2ec None admin] PATH_INFO: `/v3/projects/2bb553c7fe6e4284b56d30c136b7a041` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:39.813849 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a36f88-f5f0-49bd-a513-227bb0a6b2ec None admin] RBAC: Authorizing `identity:get_project(project_id=2bb553c7fe6e4284b56d30c136b7a041)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:39.815484 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a36f88-f5f0-49bd-a513-227bb0a6b2ec None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:39.817743 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 108/207] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:18:39 2022] GET /identity/v3/projects/2bb553c7fe6e4284b56d30c136b7a041 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:39.824185 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b8dc1-0119-4a14-bdbd-701fb536a1bc None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:39.956491 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b8dc1-0119-4a14-bdbd-701fb536a1bc None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:39.995261 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972cea2d-b34a-48be-a222-e765f3385e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:39.995992 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972cea2d-b34a-48be-a222-e765f3385e37 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:39.996211 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972cea2d-b34a-48be-a222-e765f3385e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:39.996497 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972cea2d-b34a-48be-a222-e765f3385e37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:39.997328 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972cea2d-b34a-48be-a222-e765f3385e37 None admin] RBAC: Adding query filter params (role.id=c2f1748f5cc24b8181c02f29cfea7aaf, scope.project.id=2bb553c7fe6e4284b56d30c136b7a041, user.id=1f94383c11514c06966b43e65c85d8f3) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:39.997640 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972cea2d-b34a-48be-a222-e765f3385e37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:39.999079 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972cea2d-b34a-48be-a222-e765f3385e37 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:40.005156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 100/208] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:18:39 2022] GET /identity/v3/role_assignments?user.id=1f94383c11514c06966b43e65c85d8f3&scope.project.id=2bb553c7fe6e4284b56d30c136b7a041&role.id=c2f1748f5cc24b8181c02f29cfea7aaf => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:40.822081 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494676c-12fd-459d-be2b-387c2809281a None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:40.823100 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494676c-12fd-459d-be2b-387c2809281a None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:40.823595 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494676c-12fd-459d-be2b-387c2809281a None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:40.825206 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 109/209] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:18:40.830528 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd3b05-719b-48c7-9a11-65a114c54a5c None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:40.831076 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd3b05-719b-48c7-9a11-65a114c54a5c None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:40.831546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd3b05-719b-48c7-9a11-65a114c54a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:40.836306 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3f006-1677-41db-bf24-cb2864609acc None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:40.836841 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3f006-1677-41db-bf24-cb2864609acc None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:40.837343 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3f006-1677-41db-bf24-cb2864609acc None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:40.839128 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 110/210] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:18:40.844257 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22040dac-2c6c-41c0-9c09-f4f0f0fbea4c None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:40.844761 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22040dac-2c6c-41c0-9c09-f4f0f0fbea4c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:40.845237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22040dac-2c6c-41c0-9c09-f4f0f0fbea4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:40.845428 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:40.851745 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-0fbd3b05-719b-48c7-9a11-65a114c54a5c None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:40.854578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-0fbd3b05-719b-48c7-9a11-65a114c54a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:40.863558 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-22040dac-2c6c-41c0-9c09-f4f0f0fbea4c None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:40.865036 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-22040dac-2c6c-41c0-9c09-f4f0f0fbea4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:40.880046 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:40.937215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 101/211] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:40.938508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352eaf3-ad97-406e-8362-153db8308fb0 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:40.938810 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352eaf3-ad97-406e-8362-153db8308fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:40.939092 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 111/212] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 12:18:40.945024 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352eaf3-ad97-406e-8362-153db8308fb0 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:40.945897 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 102/213] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:40.947371 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb924c-19ad-489f-86ea-50199234d103 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:40.947667 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb924c-19ad-489f-86ea-50199234d103 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:40.947952 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb924c-19ad-489f-86ea-50199234d103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:40.960938 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352eaf3-ad97-406e-8362-153db8308fb0 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:40.965536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-6acb924c-19ad-489f-86ea-50199234d103 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:40.966734 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-6acb924c-19ad-489f-86ea-50199234d103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:41.041267 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 112/214] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 12:18:41.053164 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb924c-19ad-489f-86ea-50199234d103 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:41.079350 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352eaf3-ad97-406e-8362-153db8308fb0 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:41.117066 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01177ebf-5ca8-4052-8ef9-5538f090c13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:41.117858 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01177ebf-5ca8-4052-8ef9-5538f090c13b None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:41.118169 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01177ebf-5ca8-4052-8ef9-5538f090c13b None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:41.118454 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01177ebf-5ca8-4052-8ef9-5538f090c13b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:41.121566 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:41.131098 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01177ebf-5ca8-4052-8ef9-5538f090c13b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:41.136688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:41.139346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-01177ebf-5ca8-4052-8ef9-5538f090c13b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 12:18:41.139346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:41.139346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:41.139346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:41.139346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:41.139346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:41.139346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:41.139346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:41.139346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:41.139346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:41.139346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:41.139346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:41.139346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:41.139346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:41.139346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:41.139346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:41.140271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:41.140271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:41.140271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:41.140271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:41.140271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:41.140271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:41.140271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:41.140271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:41.140271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:41.140271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:41.140271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:41.140271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:18:41.140271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:41.140271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:41.140271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:41.140271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:41.144744 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:41.144744 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:41.144744 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:41.144744 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:41.144744 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:41.144744 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:41.144744 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:41.144744 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:41.144744 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:41.144744 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:41.144744 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 12:18:41.144744 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:18:41.144744 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 103/215] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:18:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:18:41.144744 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01177ebf-5ca8-4052-8ef9-5538f090c13b None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:41.167009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb924c-19ad-489f-86ea-50199234d103 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:41.204751 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33572896-90ba-4e7e-bca9-90487b3fab1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:41.205554 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33572896-90ba-4e7e-bca9-90487b3fab1e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:41.205841 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33572896-90ba-4e7e-bca9-90487b3fab1e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:41.206235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33572896-90ba-4e7e-bca9-90487b3fab1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:41.219960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33572896-90ba-4e7e-bca9-90487b3fab1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:41.228655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-33572896-90ba-4e7e-bca9-90487b3fab1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:18:41.228655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:41.228655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:41.228655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:41.228655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:41.228655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:41.228655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:41.228655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:41.228655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:41.228655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:41.228655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:41.228655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:41.228655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:41.228655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:41.228655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:41.228655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:41.229564 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:41.229564 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:41.229564 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:41.229564 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:41.229564 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:41.229564 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:41.229564 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:41.229564 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:41.229564 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:41.229564 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:41.229564 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:41.229564 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:18:41.229564 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:41.229564 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:41.229564 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:41.229564 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:41.235438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:41.235438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:41.235438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:41.235438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:41.235438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:41.235438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:41.235438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:41.235438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:41.235438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:41.235438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:41.235438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:18:41.235438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:18:41.235438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 113/216] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:18:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:41.235438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33572896-90ba-4e7e-bca9-90487b3fab1e None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:41.257567 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01177ebf-5ca8-4052-8ef9-5538f090c13b None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:41.292346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b71e3ab-b7c4-478e-a2fe-bc7fef98d73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:41.293176 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71e3ab-b7c4-478e-a2fe-bc7fef98d73d None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:41.293440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71e3ab-b7c4-478e-a2fe-bc7fef98d73d None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:41.293720 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71e3ab-b7c4-478e-a2fe-bc7fef98d73d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:41.294972 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b71e3ab-b7c4-478e-a2fe-bc7fef98d73d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:41.296524 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b71e3ab-b7c4-478e-a2fe-bc7fef98d73d None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:41.298218 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 104/217] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:18:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:41.299039 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b71e3ab-b7c4-478e-a2fe-bc7fef98d73d None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:41.353104 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33572896-90ba-4e7e-bca9-90487b3fab1e None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:41.369454 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:41.391274 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45ff15-e963-4031-b3df-4c7e10769d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:41.391955 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45ff15-e963-4031-b3df-4c7e10769d96 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:41.392278 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45ff15-e963-4031-b3df-4c7e10769d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:41.392552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45ff15-e963-4031-b3df-4c7e10769d96 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:41.393307 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c45ff15-e963-4031-b3df-4c7e10769d96 None admin] RBAC: Adding query filter params (name=member) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:41.393618 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c45ff15-e963-4031-b3df-4c7e10769d96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:41.395016 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c45ff15-e963-4031-b3df-4c7e10769d96 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:41.401201 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:41.410417 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 114/218] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 12:18:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:41.410697 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b71e3ab-b7c4-478e-a2fe-bc7fef98d73d None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:41.411257 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45ff15-e963-4031-b3df-4c7e10769d96 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:41.447845 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8a965-4d59-4b2c-9b2f-87aa148d949e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:41.448560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8a965-4d59-4b2c-9b2f-87aa148d949e None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:41.448869 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8a965-4d59-4b2c-9b2f-87aa148d949e None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:41.449133 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8a965-4d59-4b2c-9b2f-87aa148d949e None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:41.449894 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff8a965-4d59-4b2c-9b2f-87aa148d949e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:41.450210 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff8a965-4d59-4b2c-9b2f-87aa148d949e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:41.451608 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff8a965-4d59-4b2c-9b2f-87aa148d949e None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:41.464043 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 105/219] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:18:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:41.464043 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8a965-4d59-4b2c-9b2f-87aa148d949e None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:41.525472 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45ff15-e963-4031-b3df-4c7e10769d96 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:41.558495 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990924d2-9bf6-4cfb-9bea-8ed8c90e51eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:41.559159 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990924d2-9bf6-4cfb-9bea-8ed8c90e51eb None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:41.559447 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990924d2-9bf6-4cfb-9bea-8ed8c90e51eb None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:41.559726 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990924d2-9bf6-4cfb-9bea-8ed8c90e51eb None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:41.560472 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990924d2-9bf6-4cfb-9bea-8ed8c90e51eb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:41.561909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990924d2-9bf6-4cfb-9bea-8ed8c90e51eb None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:41.566078 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8a965-4d59-4b2c-9b2f-87aa148d949e None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:41.582587 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 115/220] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 12:18:41 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:18:41.582587 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990924d2-9bf6-4cfb-9bea-8ed8c90e51eb None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:41.596618 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:41.605843 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197101c-90f5-4a74-af61-03eb476267b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:41.606654 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197101c-90f5-4a74-af61-03eb476267b4 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:41.606944 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197101c-90f5-4a74-af61-03eb476267b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:41.607227 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197101c-90f5-4a74-af61-03eb476267b4 None admin] PATH_INFO: `/v3/users/7b916ae1d63941339371998f5fa4e010` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:41.624837 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1197101c-90f5-4a74-af61-03eb476267b4 None admin] RBAC: Authorizing `identity:get_user(user_id=7b916ae1d63941339371998f5fa4e010)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:41.626349 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1197101c-90f5-4a74-af61-03eb476267b4 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:41.628208 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 106/221] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:41 2022] GET /identity/v3/users/7b916ae1d63941339371998f5fa4e010 => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:41.629468 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92b726-4146-49ff-a7ab-53efe78ae2f2 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:41.629755 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92b726-4146-49ff-a7ab-53efe78ae2f2 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:41.630031 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92b726-4146-49ff-a7ab-53efe78ae2f2 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:41.632492 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 107/222] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:41.634657 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92b726-4146-49ff-a7ab-53efe78ae2f2 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:41.669356 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:41.722120 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990924d2-9bf6-4cfb-9bea-8ed8c90e51eb None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:41.752778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f92b726-4146-49ff-a7ab-53efe78ae2f2 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:41.760659 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ecf289-8d45-4211-b064-2a42e5d83e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:41.761618 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ecf289-8d45-4211-b064-2a42e5d83e16 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:41.761835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ecf289-8d45-4211-b064-2a42e5d83e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:41.762110 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ecf289-8d45-4211-b064-2a42e5d83e16 None admin] PATH_INFO: `/v3/users/f17a529958ca41cab7529a7d68f90ee8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:41.763321 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ecf289-8d45-4211-b064-2a42e5d83e16 None admin] RBAC: Authorizing `identity:get_user(user_id=f17a529958ca41cab7529a7d68f90ee8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:41.765449 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ecf289-8d45-4211-b064-2a42e5d83e16 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:41.766778 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 116/223] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:41 2022] GET /identity/v3/users/f17a529958ca41cab7529a7d68f90ee8 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:41.767733 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fefb336-2874-4ac4-87ce-427a85973bd3 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:41.768017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fefb336-2874-4ac4-87ce-427a85973bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:41.772970 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fefb336-2874-4ac4-87ce-427a85973bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:41.791367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-7fefb336-2874-4ac4-87ce-427a85973bd3 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:41.792601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-7fefb336-2874-4ac4-87ce-427a85973bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:41.796509 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61a666-7f35-4aec-bf56-834aa1acc01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:41.797166 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61a666-7f35-4aec-bf56-834aa1acc01c None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:41.797454 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61a666-7f35-4aec-bf56-834aa1acc01c None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:41.797736 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61a666-7f35-4aec-bf56-834aa1acc01c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:41.856496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:41.871222 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 117/224] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:41.871941 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fefb336-2874-4ac4-87ce-427a85973bd3 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:41.877597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc61a666-7f35-4aec-bf56-834aa1acc01c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:41.879011 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc61a666-7f35-4aec-bf56-834aa1acc01c None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:41.937929 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:41.995904 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fefb336-2874-4ac4-87ce-427a85973bd3 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:42.008928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 108/225] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:18:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 12:18:42.010278 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61a666-7f35-4aec-bf56-834aa1acc01c None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:42.045874 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fa493-1006-48ce-8d30-e44300e26ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:42.047503 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fa493-1006-48ce-8d30-e44300e26ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:42.047929 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fa493-1006-48ce-8d30-e44300e26ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:42.048530 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fa493-1006-48ce-8d30-e44300e26ea4 None admin] PATH_INFO: `/v3/projects/49c3998144de4918b868b834ea2918dc` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:42.050362 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366fa493-1006-48ce-8d30-e44300e26ea4 None admin] RBAC: Authorizing `identity:get_project(project_id=49c3998144de4918b868b834ea2918dc)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:42.051976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366fa493-1006-48ce-8d30-e44300e26ea4 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:42.054939 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 118/226] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:18:41 2022] GET /identity/v3/projects/49c3998144de4918b868b834ea2918dc => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:42.060195 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fa493-1006-48ce-8d30-e44300e26ea4 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:42.110234 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:42.123576 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61a666-7f35-4aec-bf56-834aa1acc01c None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:42.162883 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b5859-1e7f-418f-9bf5-28e1e0f43125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:42.165202 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b5859-1e7f-418f-9bf5-28e1e0f43125 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:42.165987 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b5859-1e7f-418f-9bf5-28e1e0f43125 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:42.166407 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b5859-1e7f-418f-9bf5-28e1e0f43125 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:42.181772 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b5859-1e7f-418f-9bf5-28e1e0f43125 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:42.186487 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fa493-1006-48ce-8d30-e44300e26ea4 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:42.192764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-ac2b5859-1e7f-418f-9bf5-28e1e0f43125 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 12:18:42.192764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:42.192764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:42.192764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:42.192764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:42.192764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:42.192764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:42.192764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:42.192764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:42.192764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:42.192764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:42.192764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:42.192764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:42.192764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:42.192764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:42.192764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:42.193738 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:42.193738 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:42.193738 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:42.193738 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:42.193738 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:42.193738 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:42.193738 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:42.193738 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:42.193738 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:42.193738 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:42.193738 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:42.193738 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:18:42.193738 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:42.193738 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:42.193738 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:42.193738 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:42.195232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:42.195232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:42.195232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:42.195232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:42.195232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:42.195232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:42.195232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:42.195232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:42.195232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:42.195232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:42.195232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 12:18:42.195232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:18:42.198681 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 109/227] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 12:18:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:42.199750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b5859-1e7f-418f-9bf5-28e1e0f43125 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:42.214315 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:42.234368 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b035d-0c97-465a-8400-2cfb50b9fcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:42.235248 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b035d-0c97-465a-8400-2cfb50b9fcbe None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:42.235536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b035d-0c97-465a-8400-2cfb50b9fcbe None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:42.235814 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b035d-0c97-465a-8400-2cfb50b9fcbe None admin] PATH_INFO: `/v3/projects/49c3998144de4918b868b834ea2918dc` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:42.237066 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b035d-0c97-465a-8400-2cfb50b9fcbe None admin] RBAC: Authorizing `identity:get_project(project_id=49c3998144de4918b868b834ea2918dc)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:42.238839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b035d-0c97-465a-8400-2cfb50b9fcbe None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:42.242429 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 119/228] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:18:42 2022] GET /identity/v3/projects/49c3998144de4918b868b834ea2918dc => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:42.243474 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b035d-0c97-465a-8400-2cfb50b9fcbe None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:42.340363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b5859-1e7f-418f-9bf5-28e1e0f43125 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:42.379125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b035d-0c97-465a-8400-2cfb50b9fcbe None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:42.385696 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27532dcf-9c1c-4f03-9a9e-0617cc96a79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:42.386931 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27532dcf-9c1c-4f03-9a9e-0617cc96a79f None admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:42.387408 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27532dcf-9c1c-4f03-9a9e-0617cc96a79f None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:42.387875 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27532dcf-9c1c-4f03-9a9e-0617cc96a79f None admin] PATH_INFO: `/v3/projects/49c3998144de4918b868b834ea2918dc/users/f17a529958ca41cab7529a7d68f90ee8/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:42.391220 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27532dcf-9c1c-4f03-9a9e-0617cc96a79f None admin] RBAC: Authorizing `identity:create_grant(project_id=49c3998144de4918b868b834ea2918dc, user_id=f17a529958ca41cab7529a7d68f90ee8, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:42.393611 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27532dcf-9c1c-4f03-9a9e-0617cc96a79f None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:42.412518 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 110/229] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 12:18:42 2022] PUT /identity/v3/projects/49c3998144de4918b868b834ea2918dc/users/f17a529958ca41cab7529a7d68f90ee8/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:18:42.417606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27532dcf-9c1c-4f03-9a9e-0617cc96a79f None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:42.426962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:42.431927 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b450070-d8f7-4a9b-bf9b-d507dd65ed38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:42.432661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b450070-d8f7-4a9b-bf9b-d507dd65ed38 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:42.432950 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b450070-d8f7-4a9b-bf9b-d507dd65ed38 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:42.435738 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b450070-d8f7-4a9b-bf9b-d507dd65ed38 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:42.437786 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b450070-d8f7-4a9b-bf9b-d507dd65ed38 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:42.438121 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b450070-d8f7-4a9b-bf9b-d507dd65ed38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:42.442306 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b450070-d8f7-4a9b-bf9b-d507dd65ed38 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:42.460827 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 120/230] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:18:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:42.470509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b450070-d8f7-4a9b-bf9b-d507dd65ed38 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:42.580452 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:42.656341 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b450070-d8f7-4a9b-bf9b-d507dd65ed38 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:42.660687 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27532dcf-9c1c-4f03-9a9e-0617cc96a79f None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:42.705057 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ca8c0-cb70-452b-b2ec-19de7a1a89f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:42.706059 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ca8c0-cb70-452b-b2ec-19de7a1a89f4 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:42.706870 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ca8c0-cb70-452b-b2ec-19de7a1a89f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:42.707362 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ca8c0-cb70-452b-b2ec-19de7a1a89f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:42.708440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ca8c0-cb70-452b-b2ec-19de7a1a89f4 None admin] RBAC: Adding query filter params (role.id=504a45d998f0464893c90fa5429275a8, scope.project.id=49c3998144de4918b868b834ea2918dc, user.id=7b916ae1d63941339371998f5fa4e010) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:42.708950 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ca8c0-cb70-452b-b2ec-19de7a1a89f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:42.711516 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ca8c0-cb70-452b-b2ec-19de7a1a89f4 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:42.711726 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1eb67e-7103-4329-bc1d-e24eae4c8acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:42.712663 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1eb67e-7103-4329-bc1d-e24eae4c8acc None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:42.712954 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1eb67e-7103-4329-bc1d-e24eae4c8acc None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:42.713237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1eb67e-7103-4329-bc1d-e24eae4c8acc None admin] PATH_INFO: `/v3/users/f17a529958ca41cab7529a7d68f90ee8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:42.714769 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1eb67e-7103-4329-bc1d-e24eae4c8acc None admin] RBAC: Authorizing `identity:get_user(user_id=f17a529958ca41cab7529a7d68f90ee8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:42.716913 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1eb67e-7103-4329-bc1d-e24eae4c8acc None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:42.718952 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 121/231] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:42 2022] GET /identity/v3/users/f17a529958ca41cab7529a7d68f90ee8 => generated 283 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:42.719571 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 111/232] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:18:42 2022] GET /identity/v3/role_assignments?user.id=7b916ae1d63941339371998f5fa4e010&scope.project.id=49c3998144de4918b868b834ea2918dc&role.id=504a45d998f0464893c90fa5429275a8 => generated 259 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:42.725994 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1eb67e-7103-4329-bc1d-e24eae4c8acc None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:42.851631 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1eb67e-7103-4329-bc1d-e24eae4c8acc None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:42.873781 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:42.889516 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f87626-d5f4-469f-973c-a6d38b44fe82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:42.890398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f87626-d5f4-469f-973c-a6d38b44fe82 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:42.890683 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f87626-d5f4-469f-973c-a6d38b44fe82 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:42.890961 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f87626-d5f4-469f-973c-a6d38b44fe82 None admin] PATH_INFO: `/v3/projects/49c3998144de4918b868b834ea2918dc` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:42.892211 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f87626-d5f4-469f-973c-a6d38b44fe82 None admin] RBAC: Authorizing `identity:get_project(project_id=49c3998144de4918b868b834ea2918dc)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:42.898686 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f87626-d5f4-469f-973c-a6d38b44fe82 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:42.900643 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 122/233] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:18:42 2022] GET /identity/v3/projects/49c3998144de4918b868b834ea2918dc => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:42.906653 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ca8c0-cb70-452b-b2ec-19de7a1a89f4 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:43.013046 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:43.070737 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ca8c0-cb70-452b-b2ec-19de7a1a89f4 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:43.129468 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57f716-f88d-4744-bae9-d49689c05363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:43.130389 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57f716-f88d-4744-bae9-d49689c05363 None admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:43.130680 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57f716-f88d-4744-bae9-d49689c05363 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:43.130960 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57f716-f88d-4744-bae9-d49689c05363 None admin] PATH_INFO: `/v3/projects/49c3998144de4918b868b834ea2918dc/users/f17a529958ca41cab7529a7d68f90ee8/roles/2d4732dacad6444e86d7fee3514afefb` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:43.134012 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57f716-f88d-4744-bae9-d49689c05363 None admin] RBAC: Authorizing `identity:create_grant(project_id=49c3998144de4918b868b834ea2918dc, user_id=f17a529958ca41cab7529a7d68f90ee8, role_id=2d4732dacad6444e86d7fee3514afefb)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:43.135483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57f716-f88d-4744-bae9-d49689c05363 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:43.155916 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 112/234] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 12:18:42 2022] PUT /identity/v3/projects/49c3998144de4918b868b834ea2918dc/users/f17a529958ca41cab7529a7d68f90ee8/roles/2d4732dacad6444e86d7fee3514afefb => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:18:44.283282 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374792a0-44bb-4522-a4a4-81dd35e8eea2 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:44.284116 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374792a0-44bb-4522-a4a4-81dd35e8eea2 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:44.284599 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374792a0-44bb-4522-a4a4-81dd35e8eea2 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:44.286408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 123/235] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:44.291759 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324fa2c5-8ebb-4efb-93dd-69e7c0d7800c None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:44.292562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324fa2c5-8ebb-4efb-93dd-69e7c0d7800c None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:44.292992 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324fa2c5-8ebb-4efb-93dd-69e7c0d7800c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:44.313718 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-324fa2c5-8ebb-4efb-93dd-69e7c0d7800c None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:44.315198 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-324fa2c5-8ebb-4efb-93dd-69e7c0d7800c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:44.405504 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 113/236] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:44.450557 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374792a0-44bb-4522-a4a4-81dd35e8eea2 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:44.461899 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a32c8e-eec7-4aad-adb7-44ee9b5c2ed6 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:44.464018 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a32c8e-eec7-4aad-adb7-44ee9b5c2ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:44.464521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a32c8e-eec7-4aad-adb7-44ee9b5c2ed6 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:44.466775 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 114/237] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:44 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:44.473579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a462e-f1e1-4510-bb41-cdc49269ce67 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:44.474099 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a462e-f1e1-4510-bb41-cdc49269ce67 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:44.474518 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170a462e-f1e1-4510-bb41-cdc49269ce67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:44.509156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-170a462e-f1e1-4510-bb41-cdc49269ce67 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:44.510573 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-170a462e-f1e1-4510-bb41-cdc49269ce67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:44.570883 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374792a0-44bb-4522-a4a4-81dd35e8eea2 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:44.606123 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:44.629087 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 115/238] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 12:18:44.631072 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b1add-0c13-4d0b-bd27-b21620df2ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:44.632624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b1add-0c13-4d0b-bd27-b21620df2ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:44.633053 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b1add-0c13-4d0b-bd27-b21620df2ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:44.633476 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b1add-0c13-4d0b-bd27-b21620df2ab8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:44.643977 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036b1add-0c13-4d0b-bd27-b21620df2ab8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:44.651963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-036b1add-0c13-4d0b-bd27-b21620df2ab8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 12:18:44.651963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:44.651963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:44.651963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:44.651963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:44.651963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:44.651963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:44.651963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:44.651963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:44.651963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:44.651963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:44.651963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:44.651963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:44.651963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:44.651963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:44.651963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:44.653041 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:44.653041 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:44.653041 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:44.653041 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:44.653041 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:44.653041 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:44.653041 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:44.653041 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:44.653041 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:44.653041 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:44.653041 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:44.653041 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:18:44.653041 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:44.653041 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:44.653041 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:44.653041 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:44.654042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:44.654042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:44.654042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:44.654042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:44.654042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:44.654042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:44.654042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:44.654042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:44.654042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:44.654042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:44.654042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 12:18:44.654042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:18:44.656045 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 124/239] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:18:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:44.661894 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a462e-f1e1-4510-bb41-cdc49269ce67 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:44.675174 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b1add-0c13-4d0b-bd27-b21620df2ab8 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:44.766531 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:44.838915 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036b1add-0c13-4d0b-bd27-b21620df2ab8 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:44.839355 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170a462e-f1e1-4510-bb41-cdc49269ce67 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:44.877719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb6ff4a-eef7-41ce-ac80-ea9b95f139d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:44.879035 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6ff4a-eef7-41ce-ac80-ea9b95f139d0 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:44.882678 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6ff4a-eef7-41ce-ac80-ea9b95f139d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:44.883198 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6ff4a-eef7-41ce-ac80-ea9b95f139d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:44.884694 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb6ff4a-eef7-41ce-ac80-ea9b95f139d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:44.886689 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb6ff4a-eef7-41ce-ac80-ea9b95f139d0 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:44.889637 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 125/240] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:18:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:44.896449 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb6ff4a-eef7-41ce-ac80-ea9b95f139d0 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:44.932882 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f70db-a748-4a8d-802a-c45d76bae348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:44.933572 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f70db-a748-4a8d-802a-c45d76bae348 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:44.933857 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f70db-a748-4a8d-802a-c45d76bae348 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:44.934136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f70db-a748-4a8d-802a-c45d76bae348 None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:44.934905 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f70db-a748-4a8d-802a-c45d76bae348 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:44.935380 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f70db-a748-4a8d-802a-c45d76bae348 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:44.936936 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f70db-a748-4a8d-802a-c45d76bae348 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:44.955688 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 116/241] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 12:18:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:44.972079 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f70db-a748-4a8d-802a-c45d76bae348 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:45.009878 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb6ff4a-eef7-41ce-ac80-ea9b95f139d0 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:45.036586 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:45.055957 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f4a30-3e22-4d7a-9530-b781f7af38e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:45.056878 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f4a30-3e22-4d7a-9530-b781f7af38e2 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:45.057341 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f4a30-3e22-4d7a-9530-b781f7af38e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:45.057830 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f4a30-3e22-4d7a-9530-b781f7af38e2 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:45.058795 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f4a30-3e22-4d7a-9530-b781f7af38e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:45.060623 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f4a30-3e22-4d7a-9530-b781f7af38e2 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:45.093690 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 126/242] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 12:18:44 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:18:45.111070 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:45.185364 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f70db-a748-4a8d-802a-c45d76bae348 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:45.224670 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7ca7b-f9dc-4946-817c-8c01b2060fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:45.226432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7ca7b-f9dc-4946-817c-8c01b2060fef None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:45.226864 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7ca7b-f9dc-4946-817c-8c01b2060fef None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:45.227287 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7ca7b-f9dc-4946-817c-8c01b2060fef None admin] PATH_INFO: `/v3/users/24006dd6621041738cda555fa3ae0283` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:45.243257 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7ca7b-f9dc-4946-817c-8c01b2060fef None admin] RBAC: Authorizing `identity:get_user(user_id=24006dd6621041738cda555fa3ae0283)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:45.245021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7ca7b-f9dc-4946-817c-8c01b2060fef None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:45.247263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 117/243] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:44 2022] GET /identity/v3/users/24006dd6621041738cda555fa3ae0283 => generated 297 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:18:45.252998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f4a30-3e22-4d7a-9530-b781f7af38e2 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:45.352981 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be61d1-e863-426e-8b79-4f831dc7a6d0 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:45.353888 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be61d1-e863-426e-8b79-4f831dc7a6d0 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:45.354339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be61d1-e863-426e-8b79-4f831dc7a6d0 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:45.355884 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 118/244] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:18:45.361886 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ac4ef-77bd-44f2-8fde-cdd8101febe7 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:45.362378 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ac4ef-77bd-44f2-8fde-cdd8101febe7 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:45.362806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ac4ef-77bd-44f2-8fde-cdd8101febe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:45.391035 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-730ac4ef-77bd-44f2-8fde-cdd8101febe7 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:45.395259 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-730ac4ef-77bd-44f2-8fde-cdd8101febe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:45.398225 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f4a30-3e22-4d7a-9530-b781f7af38e2 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:45.438322 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95e02e-4e03-4622-8a97-5ae51d28d1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:45.439235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95e02e-4e03-4622-8a97-5ae51d28d1c2 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:45.439691 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95e02e-4e03-4622-8a97-5ae51d28d1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:45.440113 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95e02e-4e03-4622-8a97-5ae51d28d1c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:45.476503 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 119/245] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:45.481531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626e166-d82c-4f1a-b948-1a921c80685f None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:45.481823 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626e166-d82c-4f1a-b948-1a921c80685f None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:45.483315 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626e166-d82c-4f1a-b948-1a921c80685f None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:45.488156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 120/246] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:45.489808 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f08b4-c20b-4578-8646-91df765e4ab0 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:45.490095 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f08b4-c20b-4578-8646-91df765e4ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:45.490375 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f08b4-c20b-4578-8646-91df765e4ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:45.494754 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:45.513010 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-f99f08b4-c20b-4578-8646-91df765e4ab0 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:45.514239 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-f99f08b4-c20b-4578-8646-91df765e4ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:45.533375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c95e02e-4e03-4622-8a97-5ae51d28d1c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:45.535101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c95e02e-4e03-4622-8a97-5ae51d28d1c2 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:45.555441 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:45.601421 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 121/247] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:45.602615 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f08b4-c20b-4578-8646-91df765e4ab0 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:45.651294 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 127/248] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:18:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:18:45.652484 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95e02e-4e03-4622-8a97-5ae51d28d1c2 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:45.737105 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f08b4-c20b-4578-8646-91df765e4ab0 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:45.752391 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:45.784985 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95e02e-4e03-4622-8a97-5ae51d28d1c2 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:45.785607 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134fcf9-5533-415e-b304-7bf130a9f3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:45.786404 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134fcf9-5533-415e-b304-7bf130a9f3e8 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:45.786691 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134fcf9-5533-415e-b304-7bf130a9f3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:45.786975 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134fcf9-5533-415e-b304-7bf130a9f3e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:45.799034 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6134fcf9-5533-415e-b304-7bf130a9f3e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:45.808365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-6134fcf9-5533-415e-b304-7bf130a9f3e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:18:45.808365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:45.808365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:45.808365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:45.808365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:45.808365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:45.808365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:45.808365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:45.808365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:45.808365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:45.808365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:45.808365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:45.808365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:45.808365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:45.808365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:45.808365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:45.809278 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:45.809278 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:45.809278 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:45.809278 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:45.809278 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:45.809278 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:45.809278 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:45.809278 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:45.809278 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:45.809278 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:45.809278 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:45.809278 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:18:45.809278 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:45.809278 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:45.809278 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:45.809278 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:45.810265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:45.810265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:45.810265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:45.810265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:45.810265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:45.810265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:45.810265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:45.810265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:45.810265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:45.810265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:45.810265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:18:45.810265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:18:45.810265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 122/249] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:18:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:45.810935 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134fcf9-5533-415e-b304-7bf130a9f3e8 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:45.825963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1f026-c1a3-4aba-b8dd-dbb14e9ebf66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:45.827052 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1f026-c1a3-4aba-b8dd-dbb14e9ebf66 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:45.827337 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1f026-c1a3-4aba-b8dd-dbb14e9ebf66 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:45.827616 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1f026-c1a3-4aba-b8dd-dbb14e9ebf66 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:45.840976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1f026-c1a3-4aba-b8dd-dbb14e9ebf66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:45.842399 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:45.852067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-e4b1f026-c1a3-4aba-b8dd-dbb14e9ebf66 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 12:18:45.852067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:45.852067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:45.852067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:45.852067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:45.852067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:45.852067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:45.852067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:45.852067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:45.852067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:45.852067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:45.852067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:45.852067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:45.852067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:45.852067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:45.852067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:45.853009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:45.853009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:45.853009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:45.853009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:45.853009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:45.853009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:45.853009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:45.853009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:45.853009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:45.853009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:45.853009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:45.853009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:18:45.853009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:45.853009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:45.853009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:45.853009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:45.854007 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:45.854007 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:45.854007 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:45.854007 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:45.854007 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:45.854007 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:45.854007 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:45.854007 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:45.854007 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:45.854007 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:45.854007 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 12:18:45.854007 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:18:45.855108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 128/250] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:18:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:45.855108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1f026-c1a3-4aba-b8dd-dbb14e9ebf66 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:45.981633 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1f026-c1a3-4aba-b8dd-dbb14e9ebf66 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:45.991550 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6134fcf9-5533-415e-b304-7bf130a9f3e8 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:46.016139 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:46.039362 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113aafb-cd66-421b-98cb-bc86d9dda674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:46.040053 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113aafb-cd66-421b-98cb-bc86d9dda674 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:46.040383 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113aafb-cd66-421b-98cb-bc86d9dda674 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:46.040665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113aafb-cd66-421b-98cb-bc86d9dda674 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:46.041429 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3113aafb-cd66-421b-98cb-bc86d9dda674 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:46.041743 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3113aafb-cd66-421b-98cb-bc86d9dda674 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:46.043933 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3113aafb-cd66-421b-98cb-bc86d9dda674 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:46.067394 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 129/251] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:18:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:46.068107 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113aafb-cd66-421b-98cb-bc86d9dda674 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:46.090639 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c1313-33df-4820-aee3-8e9abd7a2c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:46.092022 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c1313-33df-4820-aee3-8e9abd7a2c48 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:46.092566 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c1313-33df-4820-aee3-8e9abd7a2c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:46.093079 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c1313-33df-4820-aee3-8e9abd7a2c48 None admin] PATH_INFO: `/v3/projects/49c3998144de4918b868b834ea2918dc` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:46.094668 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c1313-33df-4820-aee3-8e9abd7a2c48 None admin] RBAC: Authorizing `identity:get_project(project_id=49c3998144de4918b868b834ea2918dc)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:46.096631 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c1313-33df-4820-aee3-8e9abd7a2c48 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:46.103629 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 123/252] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:18:45 2022] GET /identity/v3/projects/49c3998144de4918b868b834ea2918dc => generated 302 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:46.109503 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c1313-33df-4820-aee3-8e9abd7a2c48 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:46.188777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113aafb-cd66-421b-98cb-bc86d9dda674 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:46.213870 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:46.217934 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c1313-33df-4820-aee3-8e9abd7a2c48 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:46.235134 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc2ce4-3a36-481b-acc9-b3b18b768788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:46.236357 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc2ce4-3a36-481b-acc9-b3b18b768788 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:46.237259 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc2ce4-3a36-481b-acc9-b3b18b768788 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:46.237734 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc2ce4-3a36-481b-acc9-b3b18b768788 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:46.238718 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc2ce4-3a36-481b-acc9-b3b18b768788 None admin] RBAC: Adding query filter params (name=member) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:46.239236 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc2ce4-3a36-481b-acc9-b3b18b768788 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:46.242238 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc2ce4-3a36-481b-acc9-b3b18b768788 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:46.259170 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f0014-bafa-448e-a6ca-0ae84054059c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:46.259170 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f0014-bafa-448e-a6ca-0ae84054059c None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:46.259170 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f0014-bafa-448e-a6ca-0ae84054059c None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:46.259170 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f0014-bafa-448e-a6ca-0ae84054059c None admin] PATH_INFO: `/v3/users/f17a529958ca41cab7529a7d68f90ee8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:46.260299 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 130/253] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 12:18:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:46.260519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0f0014-bafa-448e-a6ca-0ae84054059c None admin] RBAC: Authorizing `identity:get_user(user_id=f17a529958ca41cab7529a7d68f90ee8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:46.262043 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0f0014-bafa-448e-a6ca-0ae84054059c None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:46.262312 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c88fd-284c-4297-8bc1-a6b84ac73356 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:46.262786 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c88fd-284c-4297-8bc1-a6b84ac73356 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:46.263286 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c88fd-284c-4297-8bc1-a6b84ac73356 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:46.263892 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 124/254] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:46 2022] GET /identity/v3/users/f17a529958ca41cab7529a7d68f90ee8 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:46.265512 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 131/255] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:46.266448 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c88fd-284c-4297-8bc1-a6b84ac73356 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:46.269060 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f0014-bafa-448e-a6ca-0ae84054059c None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:46.344098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:46.386506 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f0014-bafa-448e-a6ca-0ae84054059c None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:46.401687 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c88fd-284c-4297-8bc1-a6b84ac73356 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:46.422939 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d68434-ff9f-4f32-89a5-6e75117459a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:46.423599 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d68434-ff9f-4f32-89a5-6e75117459a3 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:46.423885 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d68434-ff9f-4f32-89a5-6e75117459a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:46.424164 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d68434-ff9f-4f32-89a5-6e75117459a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:46.425032 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d68434-ff9f-4f32-89a5-6e75117459a3 None admin] RBAC: Adding query filter params (role.id=c2f1748f5cc24b8181c02f29cfea7aaf, scope.project.id=49c3998144de4918b868b834ea2918dc, user.id=24006dd6621041738cda555fa3ae0283) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:46.425406 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d68434-ff9f-4f32-89a5-6e75117459a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:46.426927 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d68434-ff9f-4f32-89a5-6e75117459a3 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:46.433220 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 125/256] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:18:46 2022] GET /identity/v3/role_assignments?user.id=24006dd6621041738cda555fa3ae0283&scope.project.id=49c3998144de4918b868b834ea2918dc&role.id=c2f1748f5cc24b8181c02f29cfea7aaf => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:46.434432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ba1cb-fd92-4ea5-83c5-33e84c274cdf None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:46.434725 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ba1cb-fd92-4ea5-83c5-33e84c274cdf None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:46.435006 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ba1cb-fd92-4ea5-83c5-33e84c274cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:46.445772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27b71d-8b42-49db-92a8-ab6f544d8e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:46.446664 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27b71d-8b42-49db-92a8-ab6f544d8e3d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:46.447011 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27b71d-8b42-49db-92a8-ab6f544d8e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:46.447289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27b71d-8b42-49db-92a8-ab6f544d8e3d None admin] PATH_INFO: `/v3/users/7b916ae1d63941339371998f5fa4e010` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:46.453104 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e27b71d-8b42-49db-92a8-ab6f544d8e3d None admin] RBAC: Authorizing `identity:get_user(user_id=7b916ae1d63941339371998f5fa4e010)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:46.453729 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-916ba1cb-fd92-4ea5-83c5-33e84c274cdf None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:46.454622 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e27b71d-8b42-49db-92a8-ab6f544d8e3d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:46.455011 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-916ba1cb-fd92-4ea5-83c5-33e84c274cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:46.456729 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 132/257] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:46 2022] GET /identity/v3/users/7b916ae1d63941339371998f5fa4e010 => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:46.457630 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27b71d-8b42-49db-92a8-ab6f544d8e3d None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:46.482741 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:46.533636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 126/258] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:46.534462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ba1cb-fd92-4ea5-83c5-33e84c274cdf None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:46.642155 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27b71d-8b42-49db-92a8-ab6f544d8e3d None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:46.660078 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ba1cb-fd92-4ea5-83c5-33e84c274cdf None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:46.668479 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:46.681703 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a9fc4-c95b-4ca2-a031-36096a13d011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:46.682371 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a9fc4-c95b-4ca2-a031-36096a13d011 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:46.682654 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a9fc4-c95b-4ca2-a031-36096a13d011 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:46.682932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a9fc4-c95b-4ca2-a031-36096a13d011 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:46.717815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa273e74-1652-4be0-b1b1-810865e6280d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:46.718707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa273e74-1652-4be0-b1b1-810865e6280d None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:46.718992 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa273e74-1652-4be0-b1b1-810865e6280d None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:46.719274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa273e74-1652-4be0-b1b1-810865e6280d None admin] PATH_INFO: `/v3/projects/49c3998144de4918b868b834ea2918dc` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:46.720586 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa273e74-1652-4be0-b1b1-810865e6280d None admin] RBAC: Authorizing `identity:get_project(project_id=49c3998144de4918b868b834ea2918dc)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:46.722075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa273e74-1652-4be0-b1b1-810865e6280d None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:46.724098 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 127/259] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:18:46 2022] GET /identity/v3/projects/49c3998144de4918b868b834ea2918dc => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:46.724939 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa273e74-1652-4be0-b1b1-810865e6280d None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:46.761003 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:46.764390 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1a9fc4-c95b-4ca2-a031-36096a13d011 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:46.767760 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1a9fc4-c95b-4ca2-a031-36096a13d011 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:46.884418 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 133/260] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:18:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:18:46.885463 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a9fc4-c95b-4ca2-a031-36096a13d011 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:46.908556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa273e74-1652-4be0-b1b1-810865e6280d None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:46.914131 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:46.947709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7d0f8-8436-4281-b67c-75d5603ea596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:46.948598 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7d0f8-8436-4281-b67c-75d5603ea596 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:46.948884 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7d0f8-8436-4281-b67c-75d5603ea596 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:46.949163 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7d0f8-8436-4281-b67c-75d5603ea596 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:46.969111 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b7d0f8-8436-4281-b67c-75d5603ea596 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:46.986228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-f2b7d0f8-8436-4281-b67c-75d5603ea596 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 12:18:46.986228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:46.986228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:46.986228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:46.986228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:46.986228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:46.986228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:46.986228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:46.986228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:46.986228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:46.986228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:46.986228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:46.986228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:46.986228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:46.986228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:46.986228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:46.987424 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:46.987424 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:46.987424 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:46.987424 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:46.987424 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:46.987424 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:46.987424 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:46.987424 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:46.987424 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:46.987424 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:46.987424 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:46.987424 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:18:46.987424 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:46.987424 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:46.987424 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:46.987424 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:46.988461 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:46.988461 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:46.988461 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:46.988461 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:46.988461 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:46.988461 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:46.988461 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:46.988461 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:46.988461 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:46.988461 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:46.988461 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 12:18:46.988461 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:18:46.988461 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 128/261] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 12:18:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:46.988461 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7d0f8-8436-4281-b67c-75d5603ea596 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:47.002507 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a9fc4-c95b-4ca2-a031-36096a13d011 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:47.041147 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af4b8f-1a15-47f6-bd3b-812b787ee611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:47.042385 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af4b8f-1a15-47f6-bd3b-812b787ee611 None admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:47.042905 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af4b8f-1a15-47f6-bd3b-812b787ee611 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:47.043346 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af4b8f-1a15-47f6-bd3b-812b787ee611 None admin] PATH_INFO: `/v3/projects/49c3998144de4918b868b834ea2918dc/users/7b916ae1d63941339371998f5fa4e010/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:47.045596 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44af4b8f-1a15-47f6-bd3b-812b787ee611 None admin] RBAC: Authorizing `identity:create_grant(project_id=49c3998144de4918b868b834ea2918dc, user_id=7b916ae1d63941339371998f5fa4e010, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:47.047236 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44af4b8f-1a15-47f6-bd3b-812b787ee611 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:47.068048 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 134/262] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 12:18:46 2022] PUT /identity/v3/projects/49c3998144de4918b868b834ea2918dc/users/7b916ae1d63941339371998f5fa4e010/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:18:47.069875 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af4b8f-1a15-47f6-bd3b-812b787ee611 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:47.140723 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:47.187374 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7d0f8-8436-4281-b67c-75d5603ea596 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:47.214916 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:47.235582 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af4b8f-1a15-47f6-bd3b-812b787ee611 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:47.267311 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b5cca-d882-4b2f-be1e-16b4623d0fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:47.268511 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b5cca-d882-4b2f-be1e-16b4623d0fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:47.268953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b5cca-d882-4b2f-be1e-16b4623d0fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:47.269526 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b5cca-d882-4b2f-be1e-16b4623d0fb1 None admin] PATH_INFO: `/v3/projects/49c3998144de4918b868b834ea2918dc` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:47.273545 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b5cca-d882-4b2f-be1e-16b4623d0fb1 None admin] RBAC: Authorizing `identity:get_project(project_id=49c3998144de4918b868b834ea2918dc)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:47.275092 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b5cca-d882-4b2f-be1e-16b4623d0fb1 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:47.278911 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 129/263] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:18:46 2022] GET /identity/v3/projects/49c3998144de4918b868b834ea2918dc => generated 302 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:47.284943 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b5cca-d882-4b2f-be1e-16b4623d0fb1 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:47.285948 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3083d0-296b-432a-8929-6558d2118e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:47.286629 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3083d0-296b-432a-8929-6558d2118e26 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:47.286913 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3083d0-296b-432a-8929-6558d2118e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:47.287189 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3083d0-296b-432a-8929-6558d2118e26 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:47.288155 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3083d0-296b-432a-8929-6558d2118e26 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:47.288371 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3083d0-296b-432a-8929-6558d2118e26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:47.289897 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3083d0-296b-432a-8929-6558d2118e26 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:47.310530 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 135/264] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:18:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:47.316088 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3083d0-296b-432a-8929-6558d2118e26 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:47.436627 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b5cca-d882-4b2f-be1e-16b4623d0fb1 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:47.486224 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660eef30-3392-4f28-979e-f4987ff0e75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:47.486895 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660eef30-3392-4f28-979e-f4987ff0e75c None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:47.487181 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660eef30-3392-4f28-979e-f4987ff0e75c None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:47.487459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660eef30-3392-4f28-979e-f4987ff0e75c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:47.488392 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660eef30-3392-4f28-979e-f4987ff0e75c None admin] RBAC: Adding query filter params (role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8, scope.project.id=49c3998144de4918b868b834ea2918dc, user.id=f17a529958ca41cab7529a7d68f90ee8) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:47.488710 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660eef30-3392-4f28-979e-f4987ff0e75c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:47.490181 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660eef30-3392-4f28-979e-f4987ff0e75c None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:47.506608 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 130/265] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:18:47 2022] GET /identity/v3/role_assignments?user.id=f17a529958ca41cab7529a7d68f90ee8&scope.project.id=49c3998144de4918b868b834ea2918dc&role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:47.524526 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3083d0-296b-432a-8929-6558d2118e26 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:47.575831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0f2bd-0c47-4a6f-8dd6-d23d03cb3e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:47.576767 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0f2bd-0c47-4a6f-8dd6-d23d03cb3e10 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:47.577052 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0f2bd-0c47-4a6f-8dd6-d23d03cb3e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:47.577332 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0f2bd-0c47-4a6f-8dd6-d23d03cb3e10 None admin] PATH_INFO: `/v3/users/f17a529958ca41cab7529a7d68f90ee8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:47.578654 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf0f2bd-0c47-4a6f-8dd6-d23d03cb3e10 None admin] RBAC: Authorizing `identity:get_user(user_id=f17a529958ca41cab7529a7d68f90ee8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:47.580233 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf0f2bd-0c47-4a6f-8dd6-d23d03cb3e10 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:47.582308 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 136/266] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:47 2022] GET /identity/v3/users/f17a529958ca41cab7529a7d68f90ee8 => generated 283 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:18:47.589224 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660eef30-3392-4f28-979e-f4987ff0e75c None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:47.611894 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:47.711077 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660eef30-3392-4f28-979e-f4987ff0e75c None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:47.753395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c5b90-78f3-40c8-b9b8-12a8c4fc6a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:47.753395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c5b90-78f3-40c8-b9b8-12a8c4fc6a54 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:47.753656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c5b90-78f3-40c8-b9b8-12a8c4fc6a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:47.753882 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c5b90-78f3-40c8-b9b8-12a8c4fc6a54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:47.843939 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c5b90-78f3-40c8-b9b8-12a8c4fc6a54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:47.845862 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c5b90-78f3-40c8-b9b8-12a8c4fc6a54 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:47.875815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:47.978049 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 131/267] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:18:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:18:47.983255 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0f2bd-0c47-4a6f-8dd6-d23d03cb3e10 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:48.014195 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:48.083099 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ffe85b-75cc-4be0-8948-71092f2fb98b None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:48.083725 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ffe85b-75cc-4be0-8948-71092f2fb98b None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:48.084247 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ffe85b-75cc-4be0-8948-71092f2fb98b None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:48.085964 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 132/268] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:48.092158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3894d9a-1e02-4584-9d35-8add0dc28a20 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:48.092770 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3894d9a-1e02-4584-9d35-8add0dc28a20 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:48.093256 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3894d9a-1e02-4584-9d35-8add0dc28a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:48.114223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0f2bd-0c47-4a6f-8dd6-d23d03cb3e10 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:48.129274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-c3894d9a-1e02-4584-9d35-8add0dc28a20 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:48.130909 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-c3894d9a-1e02-4584-9d35-8add0dc28a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:48.163734 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1621523-0ee2-4f65-beb1-8acdb8fd6c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:48.165831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1621523-0ee2-4f65-beb1-8acdb8fd6c3d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:48.165831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1621523-0ee2-4f65-beb1-8acdb8fd6c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:48.165831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1621523-0ee2-4f65-beb1-8acdb8fd6c3d None admin] PATH_INFO: `/v3/projects/49c3998144de4918b868b834ea2918dc` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:48.166843 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1621523-0ee2-4f65-beb1-8acdb8fd6c3d None admin] RBAC: Authorizing `identity:get_project(project_id=49c3998144de4918b868b834ea2918dc)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:48.168443 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1621523-0ee2-4f65-beb1-8acdb8fd6c3d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:48.170521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 137/269] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:18:47 2022] GET /identity/v3/projects/49c3998144de4918b868b834ea2918dc => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:48.175838 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1621523-0ee2-4f65-beb1-8acdb8fd6c3d None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:48.266704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 133/270] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:48.300529 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:48.315359 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1621523-0ee2-4f65-beb1-8acdb8fd6c3d None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:48.329266 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3894d9a-1e02-4584-9d35-8add0dc28a20 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:48.402306 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:48.410602 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba00f3b-5661-48c8-88aa-b73bb9999112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:48.411290 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba00f3b-5661-48c8-88aa-b73bb9999112 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:48.411574 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba00f3b-5661-48c8-88aa-b73bb9999112 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:48.411855 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba00f3b-5661-48c8-88aa-b73bb9999112 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:48.412765 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba00f3b-5661-48c8-88aa-b73bb9999112 None admin] RBAC: Adding query filter params (role.id=2d4732dacad6444e86d7fee3514afefb, scope.project.id=49c3998144de4918b868b834ea2918dc, user.id=f17a529958ca41cab7529a7d68f90ee8) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:48.413188 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba00f3b-5661-48c8-88aa-b73bb9999112 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:48.415159 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba00f3b-5661-48c8-88aa-b73bb9999112 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:48.434693 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 138/271] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:18:48 2022] GET /identity/v3/role_assignments?user.id=f17a529958ca41cab7529a7d68f90ee8&scope.project.id=49c3998144de4918b868b834ea2918dc&role.id=2d4732dacad6444e86d7fee3514afefb => generated 608 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:18:48.463477 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3894d9a-1e02-4584-9d35-8add0dc28a20 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:48.503268 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad067d-e8d4-4e61-8a6e-f53481732cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:48.504098 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad067d-e8d4-4e61-8a6e-f53481732cac None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:48.504409 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad067d-e8d4-4e61-8a6e-f53481732cac None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:48.504683 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad067d-e8d4-4e61-8a6e-f53481732cac None admin] PATH_INFO: `/v3/domains/default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:48.505929 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ad067d-e8d4-4e61-8a6e-f53481732cac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:48.507950 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ad067d-e8d4-4e61-8a6e-f53481732cac None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:48.509968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 134/272] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:18:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:48.515704 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba00f3b-5661-48c8-88aa-b73bb9999112 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:48.620631 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d9880-9ced-4485-8e48-14ea84c99234 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:48.621279 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d9880-9ced-4485-8e48-14ea84c99234 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:48.621735 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d9880-9ced-4485-8e48-14ea84c99234 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:48.623475 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 135/273] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:48.628966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1639cd85-2cc7-4b10-b29f-716a50f3b50b None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:48.629552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1639cd85-2cc7-4b10-b29f-716a50f3b50b None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:48.630055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1639cd85-2cc7-4b10-b29f-716a50f3b50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:48.665505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-1639cd85-2cc7-4b10-b29f-716a50f3b50b None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:48.667205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-1639cd85-2cc7-4b10-b29f-716a50f3b50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:48.699833 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba00f3b-5661-48c8-88aa-b73bb9999112 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:48.746815 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb07b11-a8c6-49bc-8b1e-a64c692d29a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:48.747500 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb07b11-a8c6-49bc-8b1e-a64c692d29a1 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:48.747787 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb07b11-a8c6-49bc-8b1e-a64c692d29a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:48.748065 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb07b11-a8c6-49bc-8b1e-a64c692d29a1 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:48.748845 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb07b11-a8c6-49bc-8b1e-a64c692d29a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:48.750380 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb07b11-a8c6-49bc-8b1e-a64c692d29a1 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:48.758818 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 136/274] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:48.776323 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 139/275] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 12:18:48 2022] POST /identity/v3/users => generated 293 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:18:48.791313 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1639cd85-2cc7-4b10-b29f-716a50f3b50b None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:48.852010 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:48.916202 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1639cd85-2cc7-4b10-b29f-716a50f3b50b None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:48.956060 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab1557-0f30-49e0-b109-fb0158bc46bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:48.957220 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab1557-0f30-49e0-b109-fb0158bc46bc None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:48.957587 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab1557-0f30-49e0-b109-fb0158bc46bc None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:48.958000 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab1557-0f30-49e0-b109-fb0158bc46bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:48.971686 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ab1557-0f30-49e0-b109-fb0158bc46bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:48.980347 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-10ab1557-0f30-49e0-b109-fb0158bc46bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 12:18:48.980347 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:48.980347 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:48.980347 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:48.980347 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:48.980347 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:48.980347 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:48.980347 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:48.980347 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:48.980347 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:48.980347 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:48.980347 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:48.980347 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:48.980347 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:48.980347 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:48.980347 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:48.981289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:48.981289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:48.981289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:48.981289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:48.981289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:48.981289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:48.981289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:48.981289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:48.981289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:48.981289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:48.981289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:48.981289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:18:48.981289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:48.981289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:48.981289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:48.981289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:48.982176 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:48.982176 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:48.982176 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:48.982176 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:48.982176 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:48.982176 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:48.982176 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:48.982176 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:48.982176 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:48.982176 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:48.982176 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 12:18:48.982176 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:18:48.983348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 137/276] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:18:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:48.987764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb07b11-a8c6-49bc-8b1e-a64c692d29a1 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:49.001141 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:49.111335 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb07b11-a8c6-49bc-8b1e-a64c692d29a1 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:49.149745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbffb02-9b59-4608-9f0a-7066b51de07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:49.150427 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbffb02-9b59-4608-9f0a-7066b51de07a None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:49.150716 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbffb02-9b59-4608-9f0a-7066b51de07a None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:49.151003 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbffb02-9b59-4608-9f0a-7066b51de07a None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:49.151762 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbffb02-9b59-4608-9f0a-7066b51de07a None admin] RBAC: Adding query filter params (name=member) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:49.152075 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbffb02-9b59-4608-9f0a-7066b51de07a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:49.153484 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbffb02-9b59-4608-9f0a-7066b51de07a None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:49.164259 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 140/277] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 12:18:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:49.169255 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab1557-0f30-49e0-b109-fb0158bc46bc None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:49.287355 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab1557-0f30-49e0-b109-fb0158bc46bc None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:49.298878 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:49.330562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f90e53-f3ed-477a-9989-bc4a6415b6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:49.331856 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f90e53-f3ed-477a-9989-bc4a6415b6d9 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:49.332406 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f90e53-f3ed-477a-9989-bc4a6415b6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:49.332919 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f90e53-f3ed-477a-9989-bc4a6415b6d9 None admin] PATH_INFO: `/v3/users/e914496ad1d54053be8014a63887403b` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:49.347986 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f90e53-f3ed-477a-9989-bc4a6415b6d9 None admin] RBAC: Authorizing `identity:get_user(user_id=e914496ad1d54053be8014a63887403b)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:49.349804 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f90e53-f3ed-477a-9989-bc4a6415b6d9 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:49.356285 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 138/278] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:49 2022] GET /identity/v3/users/e914496ad1d54053be8014a63887403b => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:49.358307 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbffb02-9b59-4608-9f0a-7066b51de07a None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:49.450749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:49.505374 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbffb02-9b59-4608-9f0a-7066b51de07a None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:49.548071 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67abfe1-64ce-4779-91da-90367ad45160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:49.549109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67abfe1-64ce-4779-91da-90367ad45160 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:49.549598 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67abfe1-64ce-4779-91da-90367ad45160 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:49.550104 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67abfe1-64ce-4779-91da-90367ad45160 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:49.551143 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67abfe1-64ce-4779-91da-90367ad45160 None admin] RBAC: Adding query filter params (role.id=504a45d998f0464893c90fa5429275a8, user.id=e914496ad1d54053be8014a63887403b, scope.system=all) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:49.551659 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67abfe1-64ce-4779-91da-90367ad45160 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:49.553398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67abfe1-64ce-4779-91da-90367ad45160 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:49.559442 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 141/279] 10.222.0.16 () {56 vars in 1308 bytes} [Mon May 23 12:18:49 2022] GET /identity/v3/role_assignments?user.id=e914496ad1d54053be8014a63887403b&scope.system=all&role.id=504a45d998f0464893c90fa5429275a8 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:49.858078 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb4bca-d4ec-43a1-baa3-23a3fbfc774a None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:49.859091 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb4bca-d4ec-43a1-baa3-23a3fbfc774a None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:49.859679 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb4bca-d4ec-43a1-baa3-23a3fbfc774a None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:49.861791 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 139/280] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:49.868009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e9167-1a2a-4356-a1b4-f251bbb8dfbd None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:49.869037 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e9167-1a2a-4356-a1b4-f251bbb8dfbd None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:49.869524 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e9167-1a2a-4356-a1b4-f251bbb8dfbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:49.893783 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-613e9167-1a2a-4356-a1b4-f251bbb8dfbd None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:49.895404 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-613e9167-1a2a-4356-a1b4-f251bbb8dfbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:49.994430 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 142/281] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:50.029784 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb4bca-d4ec-43a1-baa3-23a3fbfc774a None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:50.138815 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb84b68-7acf-4283-b653-52e75fb1b91a None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:50.139421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb84b68-7acf-4283-b653-52e75fb1b91a None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:50.139904 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb84b68-7acf-4283-b653-52e75fb1b91a None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:50.141493 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 143/282] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:18:50.149500 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e396bb-dde9-4e8f-b2d6-171429ae23ec None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:50.150049 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e396bb-dde9-4e8f-b2d6-171429ae23ec None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:50.150544 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e396bb-dde9-4e8f-b2d6-171429ae23ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:50.177176 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-16e396bb-dde9-4e8f-b2d6-171429ae23ec None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:50.178374 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-16e396bb-dde9-4e8f-b2d6-171429ae23ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:50.188400 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb4bca-d4ec-43a1-baa3-23a3fbfc774a None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:50.252390 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765cfa4-8341-49f5-b183-1765ab73ec31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:50.253190 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765cfa4-8341-49f5-b183-1765ab73ec31 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:50.253477 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765cfa4-8341-49f5-b183-1765ab73ec31 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:50.253754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765cfa4-8341-49f5-b183-1765ab73ec31 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:50.274115 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765cfa4-8341-49f5-b183-1765ab73ec31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:50.285529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-f765cfa4-8341-49f5-b183-1765ab73ec31 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 12:18:50.285529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:50.285529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:50.285529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:50.285529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:50.285529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:50.285529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:50.285529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:50.285529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:50.285529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:50.285529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:50.285529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:50.285529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:50.285529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:50.285529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:50.285529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:50.286452 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:50.286452 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:50.286452 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:50.286452 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:50.286452 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:50.286452 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:50.286452 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:50.286452 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:50.286452 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:50.286452 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:50.286452 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:50.286452 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:18:50.286452 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:50.286452 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:50.286452 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:50.286452 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:50.287254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:50.287254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:50.287254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:50.287254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:50.287254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:50.287254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:50.287254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:50.287254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:50.287254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:50.287254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:50.287254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 12:18:50.287254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:18:50.294598 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:50.299245 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 140/283] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:18:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:50.304079 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765cfa4-8341-49f5-b183-1765ab73ec31 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:50.321783 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 144/284] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 23 12:18:50.347273 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:50.383851 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e396bb-dde9-4e8f-b2d6-171429ae23ec None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:50.464842 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765cfa4-8341-49f5-b183-1765ab73ec31 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:50.508196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe1832-8c9f-45ed-ae0d-f3ba39afa468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:50.508196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe1832-8c9f-45ed-ae0d-f3ba39afa468 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:50.508196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe1832-8c9f-45ed-ae0d-f3ba39afa468 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:50.508196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe1832-8c9f-45ed-ae0d-f3ba39afa468 None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:50.508595 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbe1832-8c9f-45ed-ae0d-f3ba39afa468 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:50.509545 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbe1832-8c9f-45ed-ae0d-f3ba39afa468 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:50.510877 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbe1832-8c9f-45ed-ae0d-f3ba39afa468 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:50.528006 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 141/285] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 12:18:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:50.534019 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe1832-8c9f-45ed-ae0d-f3ba39afa468 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:50.591832 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e396bb-dde9-4e8f-b2d6-171429ae23ec None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:50.634155 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed16b0-f017-4583-8d41-35022f0833e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:50.634950 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed16b0-f017-4583-8d41-35022f0833e2 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:50.635226 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed16b0-f017-4583-8d41-35022f0833e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:50.635496 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed16b0-f017-4583-8d41-35022f0833e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:50.656488 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eed16b0-f017-4583-8d41-35022f0833e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:50.668940 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-1eed16b0-f017-4583-8d41-35022f0833e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 12:18:50.668940 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:50.668940 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:50.668940 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:50.668940 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:50.668940 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:50.668940 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:50.668940 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:50.668940 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:50.668940 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:50.668940 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:50.668940 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:50.668940 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:50.668940 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:50.668940 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:50.668940 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:50.669824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:50.669824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:50.669824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:50.669824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:50.669824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:50.669824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:50.669824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:50.669824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:50.669824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:50.669824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:50.669824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:50.669824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:18:50.669824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:50.669824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:50.669824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:50.669824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:50.670776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:50.670776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:50.670776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:50.670776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:50.670776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:50.670776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:50.670776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:50.670776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:50.670776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:50.670776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:50.670776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 12:18:50.670776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:18:50.670776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 145/286] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:18:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:50.678270 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed16b0-f017-4583-8d41-35022f0833e2 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:50.715812 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:50.740288 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe1832-8c9f-45ed-ae0d-f3ba39afa468 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:50.766955 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:50.802480 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a4ab2-c5a6-4ce8-b7c3-179ea55297ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:50.803863 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a4ab2-c5a6-4ce8-b7c3-179ea55297ec None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:50.804152 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a4ab2-c5a6-4ce8-b7c3-179ea55297ec None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:50.804458 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a4ab2-c5a6-4ce8-b7c3-179ea55297ec None admin] PATH_INFO: `/v3/users/24006dd6621041738cda555fa3ae0283` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:50.805858 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a4ab2-c5a6-4ce8-b7c3-179ea55297ec None admin] RBAC: Authorizing `identity:get_user(user_id=24006dd6621041738cda555fa3ae0283)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:50.807306 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a4ab2-c5a6-4ce8-b7c3-179ea55297ec None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:50.808953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 142/287] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:50 2022] GET /identity/v3/users/24006dd6621041738cda555fa3ae0283 => generated 297 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:18:50.814244 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a4ab2-c5a6-4ce8-b7c3-179ea55297ec None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:50.878303 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed16b0-f017-4583-8d41-35022f0833e2 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:50.955430 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c854e3-7ba0-4394-849b-40e011a0730b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:50.956332 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c854e3-7ba0-4394-849b-40e011a0730b None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:50.956713 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c854e3-7ba0-4394-849b-40e011a0730b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:50.957088 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c854e3-7ba0-4394-849b-40e011a0730b None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:50.958306 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c854e3-7ba0-4394-849b-40e011a0730b None admin] RBAC: Adding query filter params (name=member) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:50.960582 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c854e3-7ba0-4394-849b-40e011a0730b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:50.962322 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c854e3-7ba0-4394-849b-40e011a0730b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:50.978890 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 146/288] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 12:18:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:18:50.984726 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a4ab2-c5a6-4ce8-b7c3-179ea55297ec None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:50.985322 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c854e3-7ba0-4394-849b-40e011a0730b None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:51.046944 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f5552-20d9-457b-be9e-44b0eb3f3030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:51.046944 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f5552-20d9-457b-be9e-44b0eb3f3030 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:51.046944 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f5552-20d9-457b-be9e-44b0eb3f3030 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:51.046944 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f5552-20d9-457b-be9e-44b0eb3f3030 None admin] PATH_INFO: `/v3/projects/49c3998144de4918b868b834ea2918dc` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:51.050948 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5f5552-20d9-457b-be9e-44b0eb3f3030 None admin] RBAC: Authorizing `identity:get_project(project_id=49c3998144de4918b868b834ea2918dc)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:51.050948 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5f5552-20d9-457b-be9e-44b0eb3f3030 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:51.050948 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 143/289] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:18:50 2022] GET /identity/v3/projects/49c3998144de4918b868b834ea2918dc => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:18:51.056921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f5552-20d9-457b-be9e-44b0eb3f3030 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:51.127778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:51.252780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f5552-20d9-457b-be9e-44b0eb3f3030 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:51.254544 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c854e3-7ba0-4394-849b-40e011a0730b None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:51.281663 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:51.302129 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d633bbf-89e3-4bec-9be3-fd7ea9de9a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:51.303094 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d633bbf-89e3-4bec-9be3-fd7ea9de9a71 None admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:51.304136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d633bbf-89e3-4bec-9be3-fd7ea9de9a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:51.304662 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d633bbf-89e3-4bec-9be3-fd7ea9de9a71 None admin] PATH_INFO: `/v3/projects/49c3998144de4918b868b834ea2918dc/users/24006dd6621041738cda555fa3ae0283/roles/c2f1748f5cc24b8181c02f29cfea7aaf` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:51.309049 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d633bbf-89e3-4bec-9be3-fd7ea9de9a71 None admin] RBAC: Authorizing `identity:create_grant(project_id=49c3998144de4918b868b834ea2918dc, user_id=24006dd6621041738cda555fa3ae0283, role_id=c2f1748f5cc24b8181c02f29cfea7aaf)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:51.310531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d633bbf-89e3-4bec-9be3-fd7ea9de9a71 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:51.358970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 144/290] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 12:18:51 2022] PUT /identity/v3/projects/49c3998144de4918b868b834ea2918dc/users/24006dd6621041738cda555fa3ae0283/roles/c2f1748f5cc24b8181c02f29cfea7aaf => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:18:51.373777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b5db3-bb4f-4cd7-ae81-0f6cab111eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:51.375092 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b5db3-bb4f-4cd7-ae81-0f6cab111eac None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:51.375574 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b5db3-bb4f-4cd7-ae81-0f6cab111eac None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:51.376046 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b5db3-bb4f-4cd7-ae81-0f6cab111eac None admin] PATH_INFO: `/v3/users/7b916ae1d63941339371998f5fa4e010` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:51.377658 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b5db3-bb4f-4cd7-ae81-0f6cab111eac None admin] RBAC: Authorizing `identity:get_user(user_id=7b916ae1d63941339371998f5fa4e010)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:51.379352 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b5db3-bb4f-4cd7-ae81-0f6cab111eac None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:51.385964 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 147/291] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:50 2022] GET /identity/v3/users/7b916ae1d63941339371998f5fa4e010 => generated 297 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:51.393383 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d633bbf-89e3-4bec-9be3-fd7ea9de9a71 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:51.548674 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:51.574439 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d633bbf-89e3-4bec-9be3-fd7ea9de9a71 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:51.620785 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc204a45-5d89-444f-a347-4181889d6504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:51.620785 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc204a45-5d89-444f-a347-4181889d6504 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:51.620785 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc204a45-5d89-444f-a347-4181889d6504 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:51.620785 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc204a45-5d89-444f-a347-4181889d6504 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:51.698737 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc204a45-5d89-444f-a347-4181889d6504 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:51.700252 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc204a45-5d89-444f-a347-4181889d6504 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:51.701966 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986d433-cb66-4b97-be05-4f9e85d4c913 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:51.702465 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986d433-cb66-4b97-be05-4f9e85d4c913 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:51.702938 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986d433-cb66-4b97-be05-4f9e85d4c913 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:51.704835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 148/292] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:51.710374 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf735d11-38fe-4fa6-8aad-72b6dba58d6e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:51.711025 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf735d11-38fe-4fa6-8aad-72b6dba58d6e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:51.711500 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf735d11-38fe-4fa6-8aad-72b6dba58d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:51.740231 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-cf735d11-38fe-4fa6-8aad-72b6dba58d6e None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:51.748196 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-cf735d11-38fe-4fa6-8aad-72b6dba58d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:51.788250 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:51.805273 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 145/293] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:18:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:18:51.812748 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc204a45-5d89-444f-a347-4181889d6504 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:51.902907 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 149/294] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:51.908377 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdd808-4f6a-4480-b99a-ea8e228e8aaa None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:51.908691 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdd808-4f6a-4480-b99a-ea8e228e8aaa None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:51.908976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdd808-4f6a-4480-b99a-ea8e228e8aaa None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:51.910535 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 150/295] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:51.916351 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84140c34-ad3c-4a7c-9b4f-5f8498392ab1 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:51.916642 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84140c34-ad3c-4a7c-9b4f-5f8498392ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:51.916924 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84140c34-ad3c-4a7c-9b4f-5f8498392ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:51.925096 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc204a45-5d89-444f-a347-4181889d6504 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:51.949690 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-84140c34-ad3c-4a7c-9b4f-5f8498392ab1 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:51.950880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-84140c34-ad3c-4a7c-9b4f-5f8498392ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:51.963803 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04832dc4-d6b0-4ced-84b3-c0c292a468c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:51.964712 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04832dc4-d6b0-4ced-84b3-c0c292a468c9 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:51.964996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04832dc4-d6b0-4ced-84b3-c0c292a468c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:51.965279 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04832dc4-d6b0-4ced-84b3-c0c292a468c9 None admin] PATH_INFO: `/v3/projects/49c3998144de4918b868b834ea2918dc` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:51.966566 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04832dc4-d6b0-4ced-84b3-c0c292a468c9 None admin] RBAC: Authorizing `identity:get_project(project_id=49c3998144de4918b868b834ea2918dc)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:51.968155 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04832dc4-d6b0-4ced-84b3-c0c292a468c9 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:51.969996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 146/296] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:18:51 2022] GET /identity/v3/projects/49c3998144de4918b868b834ea2918dc => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:18:51.970872 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04832dc4-d6b0-4ced-84b3-c0c292a468c9 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:52.027120 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:52.032355 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 151/297] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:52.033384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84140c34-ad3c-4a7c-9b4f-5f8498392ab1 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:52.074577 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:52.083340 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04832dc4-d6b0-4ced-84b3-c0c292a468c9 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:52.127701 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c38bd9-3090-4796-8dc6-255f8f78b161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:52.129480 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c38bd9-3090-4796-8dc6-255f8f78b161 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:52.129921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c38bd9-3090-4796-8dc6-255f8f78b161 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:52.130365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c38bd9-3090-4796-8dc6-255f8f78b161 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:52.153267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c38bd9-3090-4796-8dc6-255f8f78b161 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:52.155618 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84140c34-ad3c-4a7c-9b4f-5f8498392ab1 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:52.163594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-15c38bd9-3090-4796-8dc6-255f8f78b161 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 12:18:52.163594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:52.163594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:52.163594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:52.163594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:52.163594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:52.163594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:52.163594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:52.163594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:52.163594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:52.163594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:52.163594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:52.163594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:52.163594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:52.163594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:52.163594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:52.164649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:52.164649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:52.164649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:52.164649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:52.164649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:52.164649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:52.164649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:52.164649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:52.164649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:52.164649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:52.164649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:52.164649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:18:52.164649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:52.164649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:52.164649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:52.164649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:52.165532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:52.165532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:52.165532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:52.165532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:52.165532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:52.165532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:52.165532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:52.165532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:52.165532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:52.165532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:52.165532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 12:18:52.165532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:18:52.168255 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 147/298] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:18:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:52.169804 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c38bd9-3090-4796-8dc6-255f8f78b161 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:52.202125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d92c508-7f1a-4ec0-9131-4d8c7950ac46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:52.203156 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d92c508-7f1a-4ec0-9131-4d8c7950ac46 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:52.203630 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d92c508-7f1a-4ec0-9131-4d8c7950ac46 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:52.204148 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d92c508-7f1a-4ec0-9131-4d8c7950ac46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:52.205166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d92c508-7f1a-4ec0-9131-4d8c7950ac46 None admin] RBAC: Adding query filter params (role.id=504a45d998f0464893c90fa5429275a8, scope.project.id=49c3998144de4918b868b834ea2918dc, user.id=7b916ae1d63941339371998f5fa4e010) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:52.205697 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d92c508-7f1a-4ec0-9131-4d8c7950ac46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:52.207377 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d92c508-7f1a-4ec0-9131-4d8c7950ac46 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:52.214765 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 152/299] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:18:52 2022] GET /identity/v3/role_assignments?user.id=7b916ae1d63941339371998f5fa4e010&scope.project.id=49c3998144de4918b868b834ea2918dc&role.id=504a45d998f0464893c90fa5429275a8 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:52.215895 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d92c508-7f1a-4ec0-9131-4d8c7950ac46 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:52.280211 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c38bd9-3090-4796-8dc6-255f8f78b161 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:52.292134 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:52.316951 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3acba3-8b18-4e9f-9576-77ae649bdfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:52.317853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3acba3-8b18-4e9f-9576-77ae649bdfc9 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:52.318201 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3acba3-8b18-4e9f-9576-77ae649bdfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:52.318548 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3acba3-8b18-4e9f-9576-77ae649bdfc9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:52.320562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3acba3-8b18-4e9f-9576-77ae649bdfc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:52.322301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3acba3-8b18-4e9f-9576-77ae649bdfc9 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:52.324472 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 148/300] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:18:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:52.326214 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56157d-089e-4c80-867b-b5e480a3c7eb None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:52.326545 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56157d-089e-4c80-867b-b5e480a3c7eb None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:52.326977 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56157d-089e-4c80-867b-b5e480a3c7eb None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:52.328836 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 149/301] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:52.330365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56157d-089e-4c80-867b-b5e480a3c7eb None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:52.337774 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d92c508-7f1a-4ec0-9131-4d8c7950ac46 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:52.342828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:52.378806 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c999c-f2e4-4a2b-8b50-7765f1c24a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:52.379769 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c999c-f2e4-4a2b-8b50-7765f1c24a11 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:52.380283 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c999c-f2e4-4a2b-8b50-7765f1c24a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:52.380706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c999c-f2e4-4a2b-8b50-7765f1c24a11 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:52.382189 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c999c-f2e4-4a2b-8b50-7765f1c24a11 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:52.382649 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c999c-f2e4-4a2b-8b50-7765f1c24a11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:52.384301 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c999c-f2e4-4a2b-8b50-7765f1c24a11 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:52.394020 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 153/302] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 12:18:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:52.395631 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a9985-7ec8-41c0-aef9-3e1b752c6e85 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:52.396063 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a9985-7ec8-41c0-aef9-3e1b752c6e85 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:52.396503 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a9985-7ec8-41c0-aef9-3e1b752c6e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:52.415045 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-134a9985-7ec8-41c0-aef9-3e1b752c6e85 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:52.416603 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-134a9985-7ec8-41c0-aef9-3e1b752c6e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:52.459440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56157d-089e-4c80-867b-b5e480a3c7eb None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:52.501668 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 154/303] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:52.502621 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a9985-7ec8-41c0-aef9-3e1b752c6e85 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:52.506510 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116cd463-643c-4cac-9572-c3b2af9bfcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:52.507880 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cd463-643c-4cac-9572-c3b2af9bfcc4 None admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:52.508706 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cd463-643c-4cac-9572-c3b2af9bfcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:52.509353 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cd463-643c-4cac-9572-c3b2af9bfcc4 None admin] PATH_INFO: `/v3/groups` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:52.510249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116cd463-643c-4cac-9572-c3b2af9bfcc4 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:52.511923 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116cd463-643c-4cac-9572-c3b2af9bfcc4 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:52.535439 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 150/304] 10.222.0.16 () {60 vars in 1134 bytes} [Mon May 23 12:18:52 2022] POST /identity/v3/groups => generated 229 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:18:52.536651 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116cd463-643c-4cac-9572-c3b2af9bfcc4 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:52.597363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:52.670600 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:52.677329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116cd463-643c-4cac-9572-c3b2af9bfcc4 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:52.680044 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a9985-7ec8-41c0-aef9-3e1b752c6e85 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:52.718247 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab9a47-a91d-4bf6-93ae-ced2136cf939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:52.719465 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab9a47-a91d-4bf6-93ae-ced2136cf939 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:52.719944 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab9a47-a91d-4bf6-93ae-ced2136cf939 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:52.720459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab9a47-a91d-4bf6-93ae-ced2136cf939 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:52.728688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb61037-7c54-4a72-aa00-423d043aec5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:52.729581 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb61037-7c54-4a72-aa00-423d043aec5f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:52.729863 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb61037-7c54-4a72-aa00-423d043aec5f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:52.730135 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb61037-7c54-4a72-aa00-423d043aec5f None admin] PATH_INFO: `/v3/users/303fae1403d945889bb6dffdae1b7e52` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:52.740940 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab9a47-a91d-4bf6-93ae-ced2136cf939 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:52.748921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-3fab9a47-a91d-4bf6-93ae-ced2136cf939 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 12:18:52.748921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:52.748921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:52.748921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:52.748921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:52.748921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:52.748921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:52.748921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:52.748921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:52.748921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:52.748921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:52.748921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:52.748921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:52.748921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:52.748921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:52.748921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:52.749881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:52.749881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:52.749881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:52.749881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:52.749881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:52.749881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:52.749881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:52.749881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:52.749881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:52.749881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:52.749881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:52.749881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:18:52.749881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:52.749881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:52.749881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:52.749881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:52.750705 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:52.750705 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:52.750705 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:52.750705 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:52.750705 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:52.750705 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:52.750705 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:52.750705 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:52.750705 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:52.750705 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:52.750705 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 12:18:52.750705 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:18:52.750705 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 151/305] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:18:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:52.754781 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb61037-7c54-4a72-aa00-423d043aec5f None admin] RBAC: Authorizing `identity:get_user(user_id=303fae1403d945889bb6dffdae1b7e52)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:52.756085 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab9a47-a91d-4bf6-93ae-ced2136cf939 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:52.756599 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb61037-7c54-4a72-aa00-423d043aec5f None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:52.758475 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 155/306] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:52 2022] GET /identity/v3/users/303fae1403d945889bb6dffdae1b7e52 => generated 293 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:52.766992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb61037-7c54-4a72-aa00-423d043aec5f None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:52.887010 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:52.893860 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab9a47-a91d-4bf6-93ae-ced2136cf939 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:52.920870 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb61037-7c54-4a72-aa00-423d043aec5f None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:52.933551 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de9a5f-5e51-4b1b-9b93-3ff80a7e093b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:52.934179 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de9a5f-5e51-4b1b-9b93-3ff80a7e093b None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:52.934560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de9a5f-5e51-4b1b-9b93-3ff80a7e093b None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:52.934965 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de9a5f-5e51-4b1b-9b93-3ff80a7e093b None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:52.935808 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de9a5f-5e51-4b1b-9b93-3ff80a7e093b None admin] RBAC: Adding query filter params (name=member) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:52.936240 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de9a5f-5e51-4b1b-9b93-3ff80a7e093b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:52.937733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de9a5f-5e51-4b1b-9b93-3ff80a7e093b None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:52.950616 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 152/307] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 12:18:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:18:52.956213 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de9a5f-5e51-4b1b-9b93-3ff80a7e093b None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:52.967520 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69fcb30-bb3a-40bb-8169-9dc9173b4e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:52.968368 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fcb30-bb3a-40bb-8169-9dc9173b4e25 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:52.968746 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fcb30-bb3a-40bb-8169-9dc9173b4e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:52.969136 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fcb30-bb3a-40bb-8169-9dc9173b4e25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:52.970062 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69fcb30-bb3a-40bb-8169-9dc9173b4e25 None admin] RBAC: Adding query filter params (role.id=c2f1748f5cc24b8181c02f29cfea7aaf, user.id=303fae1403d945889bb6dffdae1b7e52, scope.system=all) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:52.970500 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69fcb30-bb3a-40bb-8169-9dc9173b4e25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:52.972037 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69fcb30-bb3a-40bb-8169-9dc9173b4e25 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:52.979354 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 156/308] 10.222.0.16 () {56 vars in 1308 bytes} [Mon May 23 12:18:52 2022] GET /identity/v3/role_assignments?user.id=303fae1403d945889bb6dffdae1b7e52&scope.system=all&role.id=c2f1748f5cc24b8181c02f29cfea7aaf => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:53.097270 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de9a5f-5e51-4b1b-9b93-3ff80a7e093b None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:53.135152 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a3ae2-521e-4a31-a497-ffb5713e2a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:53.136458 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a3ae2-521e-4a31-a497-ffb5713e2a09 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:53.136955 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a3ae2-521e-4a31-a497-ffb5713e2a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:53.137496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a3ae2-521e-4a31-a497-ffb5713e2a09 None admin] PATH_INFO: `/v3/users/e914496ad1d54053be8014a63887403b` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:53.139026 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a3ae2-521e-4a31-a497-ffb5713e2a09 None admin] RBAC: Authorizing `identity:get_user(user_id=e914496ad1d54053be8014a63887403b)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:53.141042 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a3ae2-521e-4a31-a497-ffb5713e2a09 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:53.143284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 153/309] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:52 2022] GET /identity/v3/users/e914496ad1d54053be8014a63887403b => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:53.149127 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69fcb30-bb3a-40bb-8169-9dc9173b4e25 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:53.172518 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:53.266737 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69fcb30-bb3a-40bb-8169-9dc9173b4e25 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:53.303062 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c2d3a-e053-4397-b76e-2230cebe7aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:53.304005 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c2d3a-e053-4397-b76e-2230cebe7aad None admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:53.304318 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c2d3a-e053-4397-b76e-2230cebe7aad None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:53.304602 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c2d3a-e053-4397-b76e-2230cebe7aad None admin] PATH_INFO: `/v3/system/users/e914496ad1d54053be8014a63887403b/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:53.306111 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c2d3a-e053-4397-b76e-2230cebe7aad None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e914496ad1d54053be8014a63887403b, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:53.307510 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c2d3a-e053-4397-b76e-2230cebe7aad None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:53.319711 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 157/310] 10.222.0.16 () {58 vars in 1334 bytes} [Mon May 23 12:18:53 2022] PUT /identity/v3/system/users/e914496ad1d54053be8014a63887403b/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:18:53.945949 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27282c-3486-4485-99b0-39fc0532ba2a None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:53.946354 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27282c-3486-4485-99b0-39fc0532ba2a None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:53.946528 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27282c-3486-4485-99b0-39fc0532ba2a None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:53.948102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 154/311] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:53.953500 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c01b3-d7bb-40aa-963e-9f5773a35978 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:53.953934 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c01b3-d7bb-40aa-963e-9f5773a35978 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:53.954097 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c01b3-d7bb-40aa-963e-9f5773a35978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:53.971188 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-7c3c01b3-d7bb-40aa-963e-9f5773a35978 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:53.972410 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-7c3c01b3-d7bb-40aa-963e-9f5773a35978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:54.045406 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 158/312] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:54.068604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27282c-3486-4485-99b0-39fc0532ba2a None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:54.098300 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:54.175839 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27282c-3486-4485-99b0-39fc0532ba2a None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:54.210987 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b369f-c3a4-480d-9622-cbd3b9208e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:54.211707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b369f-c3a4-480d-9622-cbd3b9208e19 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:54.211989 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b369f-c3a4-480d-9622-cbd3b9208e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:54.212301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b369f-c3a4-480d-9622-cbd3b9208e19 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:54.222618 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b369f-c3a4-480d-9622-cbd3b9208e19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:54.230838 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-1e5b369f-c3a4-480d-9622-cbd3b9208e19 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 12:18:54.230838 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:54.230838 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:54.230838 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:54.230838 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:54.230838 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:54.230838 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:54.230838 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:54.230838 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:54.230838 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:54.230838 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:54.230838 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:54.230838 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:54.230838 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:54.230838 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:54.230838 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:54.231719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:54.231719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:54.231719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:54.231719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:54.231719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:54.231719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:54.231719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:54.231719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:54.231719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:54.231719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:54.231719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:54.231719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:18:54.231719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:54.231719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:54.231719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:54.231719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:54.232573 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:54.232573 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:54.232573 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:54.232573 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:54.232573 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:54.232573 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:54.232573 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:54.232573 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:54.232573 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:54.232573 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:54.232573 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 12:18:54.232573 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:18:54.232573 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 155/313] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:18:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:54.238132 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c01b3-d7bb-40aa-963e-9f5773a35978 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:54.250708 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:54.342868 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c01b3-d7bb-40aa-963e-9f5773a35978 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:54.378064 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86111caa-4d10-46b8-b5fb-6e8dbe0ae935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:54.378735 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86111caa-4d10-46b8-b5fb-6e8dbe0ae935 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:54.379016 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86111caa-4d10-46b8-b5fb-6e8dbe0ae935 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:54.379293 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86111caa-4d10-46b8-b5fb-6e8dbe0ae935 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:54.380030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86111caa-4d10-46b8-b5fb-6e8dbe0ae935 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:54.380367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86111caa-4d10-46b8-b5fb-6e8dbe0ae935 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:54.382632 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86111caa-4d10-46b8-b5fb-6e8dbe0ae935 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:54.391500 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 159/314] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 12:18:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:54.396797 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b369f-c3a4-480d-9622-cbd3b9208e19 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:54.493096 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:54.505484 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b369f-c3a4-480d-9622-cbd3b9208e19 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:54.541454 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedbb3dc-6572-4243-ae06-c2c05c2413d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:54.542334 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbb3dc-6572-4243-ae06-c2c05c2413d3 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:54.542623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbb3dc-6572-4243-ae06-c2c05c2413d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:54.542905 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbb3dc-6572-4243-ae06-c2c05c2413d3 None admin] PATH_INFO: `/v3/users/24006dd6621041738cda555fa3ae0283` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:54.544121 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedbb3dc-6572-4243-ae06-c2c05c2413d3 None admin] RBAC: Authorizing `identity:get_user(user_id=24006dd6621041738cda555fa3ae0283)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:54.545596 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedbb3dc-6572-4243-ae06-c2c05c2413d3 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:54.547217 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 156/315] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:54 2022] GET /identity/v3/users/24006dd6621041738cda555fa3ae0283 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:54.552863 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86111caa-4d10-46b8-b5fb-6e8dbe0ae935 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:54.627476 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:54.658974 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86111caa-4d10-46b8-b5fb-6e8dbe0ae935 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:54.700087 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc2958-a58f-45e3-8ab8-69f15b365231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:54.700783 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc2958-a58f-45e3-8ab8-69f15b365231 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:54.701065 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc2958-a58f-45e3-8ab8-69f15b365231 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:54.701338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc2958-a58f-45e3-8ab8-69f15b365231 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:54.777347 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc2958-a58f-45e3-8ab8-69f15b365231 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:54.778744 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc2958-a58f-45e3-8ab8-69f15b365231 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:54.852610 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:54.883778 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 160/316] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:18:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:18:54.889136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedbb3dc-6572-4243-ae06-c2c05c2413d3 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:54.900034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d85d8-61ca-42b4-a110-59496d84d490 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:54.900360 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d85d8-61ca-42b4-a110-59496d84d490 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:54.900616 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d85d8-61ca-42b4-a110-59496d84d490 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:54.902215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 161/317] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:18:54.908419 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517f094-4410-49f2-a5a9-76e380326d95 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:54.908735 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517f094-4410-49f2-a5a9-76e380326d95 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:54.909092 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517f094-4410-49f2-a5a9-76e380326d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:54.928514 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-2517f094-4410-49f2-a5a9-76e380326d95 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:54.929867 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-2517f094-4410-49f2-a5a9-76e380326d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:55.004936 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 162/318] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 12:18:55.015108 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedbb3dc-6572-4243-ae06-c2c05c2413d3 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:55.032534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517f094-4410-49f2-a5a9-76e380326d95 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:55.051309 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9783801-390c-47d1-a9c1-a1373baa1b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:55.052300 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9783801-390c-47d1-a9c1-a1373baa1b31 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:55.052776 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9783801-390c-47d1-a9c1-a1373baa1b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:55.053119 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9783801-390c-47d1-a9c1-a1373baa1b31 None admin] PATH_INFO: `/v3/projects/49c3998144de4918b868b834ea2918dc` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:55.055137 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9783801-390c-47d1-a9c1-a1373baa1b31 None admin] RBAC: Authorizing `identity:get_project(project_id=49c3998144de4918b868b834ea2918dc)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:55.056784 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9783801-390c-47d1-a9c1-a1373baa1b31 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:55.058950 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 157/319] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:18:54 2022] GET /identity/v3/projects/49c3998144de4918b868b834ea2918dc => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:55.064406 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9783801-390c-47d1-a9c1-a1373baa1b31 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:55.092873 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:55.180664 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9783801-390c-47d1-a9c1-a1373baa1b31 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:55.182270 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517f094-4410-49f2-a5a9-76e380326d95 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:55.185241 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:55.222274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb896b-aa29-494f-b6f6-ca873992240f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:55.222974 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb896b-aa29-494f-b6f6-ca873992240f None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:55.223310 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb896b-aa29-494f-b6f6-ca873992240f None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:55.223655 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb896b-aa29-494f-b6f6-ca873992240f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:55.224693 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb896b-aa29-494f-b6f6-ca873992240f None admin] RBAC: Adding query filter params (role.id=c2f1748f5cc24b8181c02f29cfea7aaf, scope.project.id=49c3998144de4918b868b834ea2918dc, user.id=24006dd6621041738cda555fa3ae0283) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:55.225098 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb896b-aa29-494f-b6f6-ca873992240f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:55.227415 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb896b-aa29-494f-b6f6-ca873992240f None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:55.235213 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864e5ce-1f2e-441e-8947-cf3d9b9470e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:55.236294 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864e5ce-1f2e-441e-8947-cf3d9b9470e0 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:55.236740 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864e5ce-1f2e-441e-8947-cf3d9b9470e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:55.237180 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864e5ce-1f2e-441e-8947-cf3d9b9470e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:55.238589 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864e5ce-1f2e-441e-8947-cf3d9b9470e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:55.240309 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3864e5ce-1f2e-441e-8947-cf3d9b9470e0 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:55.240709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 158/320] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:18:55 2022] GET /identity/v3/role_assignments?user.id=24006dd6621041738cda555fa3ae0283&scope.project.id=49c3998144de4918b868b834ea2918dc&role.id=c2f1748f5cc24b8181c02f29cfea7aaf => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:55.242790 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 163/321] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:18:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:18:55.248912 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb896b-aa29-494f-b6f6-ca873992240f None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:55.343126 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ce385-31dc-4536-ae0f-99412c367a12 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:55.343126 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ce385-31dc-4536-ae0f-99412c367a12 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:55.343126 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ce385-31dc-4536-ae0f-99412c367a12 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:55.344755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 164/322] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:18:55.351068 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9405fa-bbf8-40e7-81ff-61b5bbfc35e6 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:55.351585 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9405fa-bbf8-40e7-81ff-61b5bbfc35e6 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:55.352050 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9405fa-bbf8-40e7-81ff-61b5bbfc35e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:55.360818 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:55.377401 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb896b-aa29-494f-b6f6-ca873992240f None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:55.379675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-7d9405fa-bbf8-40e7-81ff-61b5bbfc35e6 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:55.382444 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-7d9405fa-bbf8-40e7-81ff-61b5bbfc35e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:55.416744 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd9328-e801-4af7-8873-3c6c5acd741b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:55.417505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd9328-e801-4af7-8873-3c6c5acd741b None admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:55.417797 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd9328-e801-4af7-8873-3c6c5acd741b None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:55.418075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd9328-e801-4af7-8873-3c6c5acd741b None admin] PATH_INFO: `/v3/groups` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:55.418821 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cd9328-e801-4af7-8873-3c6c5acd741b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:55.420456 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cd9328-e801-4af7-8873-3c6c5acd741b None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:55.444205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 159/323] 10.222.0.16 () {60 vars in 1134 bytes} [Mon May 23 12:18:55 2022] POST /identity/v3/groups => generated 226 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:18:55.482785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 165/324] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 12:18:55.506392 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd9328-e801-4af7-8873-3c6c5acd741b None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:55.625155 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd9328-e801-4af7-8873-3c6c5acd741b None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:55.662068 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d378e-0162-47a9-96a6-7fe04432c09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:55.663595 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d378e-0162-47a9-96a6-7fe04432c09c None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:55.663928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d378e-0162-47a9-96a6-7fe04432c09c None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:55.664300 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d378e-0162-47a9-96a6-7fe04432c09c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:55.675168 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d378e-0162-47a9-96a6-7fe04432c09c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:55.683886 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-2f7d378e-0162-47a9-96a6-7fe04432c09c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 12:18:55.683886 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:55.683886 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:55.683886 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:55.683886 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:55.683886 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:55.683886 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:55.683886 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:55.683886 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:55.683886 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:55.683886 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:55.683886 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:55.683886 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:55.683886 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:55.683886 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:55.683886 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:55.684801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:55.684801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:55.684801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:55.684801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:55.684801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:55.684801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:55.684801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:55.684801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:55.684801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:55.684801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:55.684801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:55.684801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:18:55.684801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:55.684801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:55.684801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:55.684801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:55.685638 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:55.685638 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:55.685638 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:55.685638 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:55.685638 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:55.685638 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:55.685638 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:55.685638 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:55.685638 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:55.685638 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:55.685638 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 12:18:55.685638 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:18:55.685638 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 160/325] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:18:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:55.690945 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9405fa-bbf8-40e7-81ff-61b5bbfc35e6 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:55.711937 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91d2bb-5ece-4481-9876-8e6121281b50 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:55.712314 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91d2bb-5ece-4481-9876-8e6121281b50 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:55.712555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91d2bb-5ece-4481-9876-8e6121281b50 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:55.714061 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 161/326] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:55.719109 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7a880-c1bf-497b-b184-6163de12b8ee None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:55.719430 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7a880-c1bf-497b-b184-6163de12b8ee None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:55.719753 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b7a880-c1bf-497b-b184-6163de12b8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:55.740343 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-b5b7a880-c1bf-497b-b184-6163de12b8ee None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:55.742342 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-b5b7a880-c1bf-497b-b184-6163de12b8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:55.745342 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:55.812624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:55.815732 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9405fa-bbf8-40e7-81ff-61b5bbfc35e6 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:55.836584 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 162/327] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 12:18:55.858517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b7a880-c1bf-497b-b184-6163de12b8ee None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:55.860669 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a337d-63f2-463b-a5c8-6acdd37bcc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:55.861468 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a337d-63f2-463b-a5c8-6acdd37bcc33 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:55.861854 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a337d-63f2-463b-a5c8-6acdd37bcc33 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:55.862203 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a337d-63f2-463b-a5c8-6acdd37bcc33 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:55.863046 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a337d-63f2-463b-a5c8-6acdd37bcc33 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:55.863458 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a337d-63f2-463b-a5c8-6acdd37bcc33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:55.865031 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a337d-63f2-463b-a5c8-6acdd37bcc33 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:55.880374 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 166/328] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 12:18:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:55.882416 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a337d-63f2-463b-a5c8-6acdd37bcc33 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:55.997723 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b7a880-c1bf-497b-b184-6163de12b8ee None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:56.018300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a337d-63f2-463b-a5c8-6acdd37bcc33 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:56.050831 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b0e38-5c14-4673-a23a-0c490f81860a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:56.051895 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b0e38-5c14-4673-a23a-0c490f81860a None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:56.052253 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b0e38-5c14-4673-a23a-0c490f81860a None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:56.053955 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5b0e38-5c14-4673-a23a-0c490f81860a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:56.064940 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:56.072826 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5b0e38-5c14-4673-a23a-0c490f81860a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:56.082376 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad272eaf-038a-4750-b5dd-9517e91b69e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:56.083257 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad272eaf-038a-4750-b5dd-9517e91b69e3 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:56.083541 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad272eaf-038a-4750-b5dd-9517e91b69e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:56.083823 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad272eaf-038a-4750-b5dd-9517e91b69e3 None admin] PATH_INFO: `/v3/users/303fae1403d945889bb6dffdae1b7e52` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:56.084579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-7e5b0e38-5c14-4673-a23a-0c490f81860a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 12:18:56.084579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:56.084579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:56.084579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:56.084579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:56.084579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:56.084579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:56.084579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:56.084579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:56.084579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:56.084579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:56.084579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:56.084579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:56.084579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:56.084579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:56.084579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:56.085540 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:56.085540 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:56.085540 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:56.085540 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:56.085540 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:56.085540 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:56.085540 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:56.085540 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:56.085540 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:56.085540 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:56.085540 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:56.085540 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:18:56.085540 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:56.085540 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:56.085540 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:56.085540 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:56.086330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:56.086330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:56.086330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:56.086330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:56.086330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:56.086330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:56.086330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:56.086330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:56.086330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:56.086330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:56.086330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 12:18:56.086330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:18:56.092195 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad272eaf-038a-4750-b5dd-9517e91b69e3 None admin] RBAC: Authorizing `identity:get_user(user_id=303fae1403d945889bb6dffdae1b7e52)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:56.092195 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad272eaf-038a-4750-b5dd-9517e91b69e3 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:56.092395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 163/329] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:18:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:56.096644 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 167/330] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:55 2022] GET /identity/v3/users/303fae1403d945889bb6dffdae1b7e52 => generated 293 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:56.101526 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad272eaf-038a-4750-b5dd-9517e91b69e3 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:56.101805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b0e38-5c14-4673-a23a-0c490f81860a None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:56.142114 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:56.232841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5b0e38-5c14-4673-a23a-0c490f81860a None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:56.233756 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad272eaf-038a-4750-b5dd-9517e91b69e3 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:56.277095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e8afb-54aa-4e16-be4a-a4744a26bac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:56.278633 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e8afb-54aa-4e16-be4a-a4744a26bac7 None admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:56.278973 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d670646-bec3-46d3-99b3-0de078ce745a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:56.279354 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e8afb-54aa-4e16-be4a-a4744a26bac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:56.279642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d670646-bec3-46d3-99b3-0de078ce745a None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:56.279924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d670646-bec3-46d3-99b3-0de078ce745a None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:56.280266 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e8afb-54aa-4e16-be4a-a4744a26bac7 None admin] PATH_INFO: `/v3/system/users/303fae1403d945889bb6dffdae1b7e52/roles/c2f1748f5cc24b8181c02f29cfea7aaf` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:56.280368 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d670646-bec3-46d3-99b3-0de078ce745a None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:56.281034 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d670646-bec3-46d3-99b3-0de078ce745a None admin] RBAC: Adding query filter params (name=member) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:56.281390 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d670646-bec3-46d3-99b3-0de078ce745a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:56.282496 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93e8afb-54aa-4e16-be4a-a4744a26bac7 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=303fae1403d945889bb6dffdae1b7e52, role_id=c2f1748f5cc24b8181c02f29cfea7aaf)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:56.282773 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d670646-bec3-46d3-99b3-0de078ce745a None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:56.284320 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93e8afb-54aa-4e16-be4a-a4744a26bac7 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:56.297469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 164/331] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 12:18:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:56.301025 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 168/332] 10.222.0.16 () {58 vars in 1334 bytes} [Mon May 23 12:18:56 2022] PUT /identity/v3/system/users/303fae1403d945889bb6dffdae1b7e52/roles/c2f1748f5cc24b8181c02f29cfea7aaf => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:18:56.305036 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d670646-bec3-46d3-99b3-0de078ce745a None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:56.377839 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:56.426815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d670646-bec3-46d3-99b3-0de078ce745a None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:56.464895 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9391fa-dff0-4b8c-8379-989162c6f748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:56.465794 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9391fa-dff0-4b8c-8379-989162c6f748 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:56.466079 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9391fa-dff0-4b8c-8379-989162c6f748 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:56.466352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9391fa-dff0-4b8c-8379-989162c6f748 None admin] PATH_INFO: `/v3/users/e914496ad1d54053be8014a63887403b` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:56.467609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9391fa-dff0-4b8c-8379-989162c6f748 None admin] RBAC: Authorizing `identity:get_user(user_id=e914496ad1d54053be8014a63887403b)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:56.469205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9391fa-dff0-4b8c-8379-989162c6f748 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:56.471247 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 165/333] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:18:56 2022] GET /identity/v3/users/e914496ad1d54053be8014a63887403b => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:56.477439 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e8afb-54aa-4e16-be4a-a4744a26bac7 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:56.592819 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:56.600687 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e8afb-54aa-4e16-be4a-a4744a26bac7 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:56.639239 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882f642-3481-48e0-9e50-b46d639aed6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:56.639951 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882f642-3481-48e0-9e50-b46d639aed6c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:56.640264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882f642-3481-48e0-9e50-b46d639aed6c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:56.640542 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882f642-3481-48e0-9e50-b46d639aed6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:56.641377 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882f642-3481-48e0-9e50-b46d639aed6c None admin] RBAC: Adding query filter params (role.id=504a45d998f0464893c90fa5429275a8, user.id=e914496ad1d54053be8014a63887403b, scope.system=all) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:56.641690 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882f642-3481-48e0-9e50-b46d639aed6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:56.643137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882f642-3481-48e0-9e50-b46d639aed6c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:56.656958 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 169/334] 10.222.0.16 () {56 vars in 1308 bytes} [Mon May 23 12:18:56 2022] GET /identity/v3/role_assignments?user.id=e914496ad1d54053be8014a63887403b&scope.system=all&role.id=504a45d998f0464893c90fa5429275a8 => generated 510 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:58.509309 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59672912-51af-4552-8c68-605594d4c191 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:58.509826 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59672912-51af-4552-8c68-605594d4c191 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:58.509826 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59672912-51af-4552-8c68-605594d4c191 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:58.511372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 166/335] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:18:58.517126 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d1cd6-9ff3-4bfb-9a2b-0337d6f2ee5b None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:58.517523 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d1cd6-9ff3-4bfb-9a2b-0337d6f2ee5b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:58.517690 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d1cd6-9ff3-4bfb-9a2b-0337d6f2ee5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:58.539599 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-3e3d1cd6-9ff3-4bfb-9a2b-0337d6f2ee5b None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:58.540811 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3e3d1cd6-9ff3-4bfb-9a2b-0337d6f2ee5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:58.630679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 170/336] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:58.654016 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f9148-27d4-48d8-a03c-e6bf4c46e7ad None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:58.654339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f9148-27d4-48d8-a03c-e6bf4c46e7ad None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:58.654654 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f9148-27d4-48d8-a03c-e6bf4c46e7ad None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:58.656021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 167/337] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:18:58.676809 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399f661-d672-49ad-865f-d3e060091e28 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:58.677123 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399f661-d672-49ad-865f-d3e060091e28 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:58.677371 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399f661-d672-49ad-865f-d3e060091e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:58.682445 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f9148-27d4-48d8-a03c-e6bf4c46e7ad None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:58.698459 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-f399f661-d672-49ad-865f-d3e060091e28 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:58.699792 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-f399f661-d672-49ad-865f-d3e060091e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:18:58.778256 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 171/338] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:18:58.819291 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399f661-d672-49ad-865f-d3e060091e28 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:58.833986 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:58.888929 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f9148-27d4-48d8-a03c-e6bf4c46e7ad None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:58.947730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:58.962969 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9b0d9-8db2-48c4-9df8-b6a5741d1d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:58.963776 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9b0d9-8db2-48c4-9df8-b6a5741d1d9f None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:58.985910 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399f661-d672-49ad-865f-d3e060091e28 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:58.992931 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9b0d9-8db2-48c4-9df8-b6a5741d1d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:58.993505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9b0d9-8db2-48c4-9df8-b6a5741d1d9f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:59.028911 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9b0d9-8db2-48c4-9df8-b6a5741d1d9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:59.036648 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-b7e9b0d9-8db2-48c4-9df8-b6a5741d1d9f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 12:18:59.036648 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:59.036648 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:59.036648 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:59.036648 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:59.036648 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:59.036648 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:59.036648 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:59.036648 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:59.036648 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:59.036648 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:59.036648 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:59.036648 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:59.036648 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:59.036648 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:59.036648 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:59.037560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:59.037560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:59.037560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:59.037560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:59.037560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:59.037560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:59.037560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:59.037560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:59.037560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:59.037560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:59.037560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:59.037560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:18:59.037560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:59.037560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:59.037560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:59.037560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:59.038899 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:59.038899 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:59.038899 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:59.038899 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:59.038899 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:59.038899 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:59.038899 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:59.038899 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:59.038899 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:59.038899 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:59.038899 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 12:18:59.038899 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:18:59.044296 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 168/339] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:18:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:18:59.048627 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9b0d9-8db2-48c4-9df8-b6a5741d1d9f None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:59.060882 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691cbec3-f592-41ee-bbca-8b73e56c0188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:59.061725 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cbec3-f592-41ee-bbca-8b73e56c0188 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:59.062009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cbec3-f592-41ee-bbca-8b73e56c0188 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:59.062287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cbec3-f592-41ee-bbca-8b73e56c0188 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:59.080090 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691cbec3-f592-41ee-bbca-8b73e56c0188 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:18:59.098143 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-691cbec3-f592-41ee-bbca-8b73e56c0188 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 12:18:59.098143 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:18:59.098143 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:18:59.098143 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:18:59.098143 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:18:59.098143 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:18:59.098143 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:18:59.098143 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:18:59.098143 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:18:59.098143 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:18:59.098143 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:18:59.098143 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:18:59.098143 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:18:59.098143 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:18:59.098143 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:18:59.098143 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:18:59.099232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:18:59.099232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:18:59.099232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:18:59.099232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:18:59.099232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:18:59.099232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:18:59.099232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:18:59.099232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:18:59.099232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:18:59.099232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:18:59.099232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:18:59.099232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:18:59.099232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:18:59.099232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:18:59.099232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:18:59.099232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:18:59.100261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:18:59.100261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:18:59.100261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:18:59.100261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:18:59.100261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:18:59.100261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:18:59.100261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:18:59.100261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:18:59.100261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:18:59.100261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:18:59.100261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 12:18:59.100261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:18:59.100261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 172/340] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 12:18:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:18:59.106537 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691cbec3-f592-41ee-bbca-8b73e56c0188 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:59.246558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9b0d9-8db2-48c4-9df8-b6a5741d1d9f None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:59.272746 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:59.300101 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6177118-87e0-46f6-931b-88c09944465f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:59.301151 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6177118-87e0-46f6-931b-88c09944465f None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:59.301712 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6177118-87e0-46f6-931b-88c09944465f None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:59.302318 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6177118-87e0-46f6-931b-88c09944465f None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:59.304402 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6177118-87e0-46f6-931b-88c09944465f None admin] RBAC: Adding query filter params (name=member) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:59.304953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6177118-87e0-46f6-931b-88c09944465f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:59.306568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6177118-87e0-46f6-931b-88c09944465f None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:59.319663 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691cbec3-f592-41ee-bbca-8b73e56c0188 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:59.323856 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 169/341] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 12:18:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:18:59.329609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6177118-87e0-46f6-931b-88c09944465f None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:59.359069 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c62514-390d-4cc4-88dd-8210e4e59bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:59.359761 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c62514-390d-4cc4-88dd-8210e4e59bff None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:59.360048 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c62514-390d-4cc4-88dd-8210e4e59bff None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:59.360385 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c62514-390d-4cc4-88dd-8210e4e59bff None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:59.361269 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c62514-390d-4cc4-88dd-8210e4e59bff None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:18:59.361585 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c62514-390d-4cc4-88dd-8210e4e59bff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:59.362994 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c62514-390d-4cc4-88dd-8210e4e59bff None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:59.372822 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 173/342] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:18:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:18:59.378420 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c62514-390d-4cc4-88dd-8210e4e59bff None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:59.419124 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:59.498092 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c62514-390d-4cc4-88dd-8210e4e59bff None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:59.534430 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83fd6a0-9f5e-4e44-b39e-e875f47a039e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:59.535176 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83fd6a0-9f5e-4e44-b39e-e875f47a039e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:59.535363 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83fd6a0-9f5e-4e44-b39e-e875f47a039e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:59.535900 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83fd6a0-9f5e-4e44-b39e-e875f47a039e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:59.540473 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:59.582262 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6177118-87e0-46f6-931b-88c09944465f None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:59.613658 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83fd6a0-9f5e-4e44-b39e-e875f47a039e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:59.615292 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83fd6a0-9f5e-4e44-b39e-e875f47a039e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:59.685546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eaf8cd-1f30-4e7d-9f16-24ac9ba4bfcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:59.686225 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eaf8cd-1f30-4e7d-9f16-24ac9ba4bfcb None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:59.686526 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eaf8cd-1f30-4e7d-9f16-24ac9ba4bfcb None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:59.686811 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eaf8cd-1f30-4e7d-9f16-24ac9ba4bfcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:59.726238 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 174/343] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:18:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:18:59.732500 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83fd6a0-9f5e-4e44-b39e-e875f47a039e None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:18:59.771076 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:18:59.842319 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83fd6a0-9f5e-4e44-b39e-e875f47a039e None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:18:59.879887 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d4253-b971-4ba6-ab9f-d4855b62857b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:18:59.881153 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d4253-b971-4ba6-ab9f-d4855b62857b None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:59.882179 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d4253-b971-4ba6-ab9f-d4855b62857b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:59.882623 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d4253-b971-4ba6-ab9f-d4855b62857b None admin] PATH_INFO: `/v3/projects/2bb553c7fe6e4284b56d30c136b7a041` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:59.883981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d4253-b971-4ba6-ab9f-d4855b62857b None admin] RBAC: Authorizing `identity:get_project(project_id=2bb553c7fe6e4284b56d30c136b7a041)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:59.885848 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d4253-b971-4ba6-ab9f-d4855b62857b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:59.887719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 175/344] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:18:59 2022] GET /identity/v3/projects/2bb553c7fe6e4284b56d30c136b7a041 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:18:59.889642 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505195f-ee3a-42ce-b28d-f04b97ce42fa None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:59.890107 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505195f-ee3a-42ce-b28d-f04b97ce42fa None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:59.890528 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505195f-ee3a-42ce-b28d-f04b97ce42fa None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:59.891911 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 176/345] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:18:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:18:59.897553 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648cf93-899b-4bb0-b2cc-7d51a38e3997 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:18:59.898021 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648cf93-899b-4bb0-b2cc-7d51a38e3997 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:18:59.898445 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648cf93-899b-4bb0-b2cc-7d51a38e3997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:18:59.917079 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eaf8cd-1f30-4e7d-9f16-24ac9ba4bfcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:18:59.918204 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a648cf93-899b-4bb0-b2cc-7d51a38e3997 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:18:59.918578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eaf8cd-1f30-4e7d-9f16-24ac9ba4bfcb None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:18:59.919709 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a648cf93-899b-4bb0-b2cc-7d51a38e3997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:00.013484 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:00.019063 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 177/346] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:18:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 12:19:00.020387 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a648cf93-899b-4bb0-b2cc-7d51a38e3997 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:00.059339 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:00.102412 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 170/347] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:18:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 773 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:19:00.104084 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b1219-1cc6-462b-af85-542ad53d2095 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:00.104601 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b1219-1cc6-462b-af85-542ad53d2095 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:00.105056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b1219-1cc6-462b-af85-542ad53d2095 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:00.108986 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 171/348] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:00.110018 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b1219-1cc6-462b-af85-542ad53d2095 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:00.164670 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a648cf93-899b-4bb0-b2cc-7d51a38e3997 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:00.205500 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97801c-ee8b-49ab-9599-d3466c7375df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:00.206375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97801c-ee8b-49ab-9599-d3466c7375df None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:00.206655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97801c-ee8b-49ab-9599-d3466c7375df None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:00.206929 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97801c-ee8b-49ab-9599-d3466c7375df None admin] PATH_INFO: `/v3/groups/b14634a2a38a44db8071dd00d49ded4a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:00.218575 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97801c-ee8b-49ab-9599-d3466c7375df None admin] RBAC: Authorizing `identity:get_group(group_id=b14634a2a38a44db8071dd00d49ded4a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:00.220146 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97801c-ee8b-49ab-9599-d3466c7375df None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:00.222419 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 178/349] 10.222.0.16 () {56 vars in 1180 bytes} [Mon May 23 12:19:00 2022] GET /identity/v3/groups/b14634a2a38a44db8071dd00d49ded4a => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:00.223343 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97801c-ee8b-49ab-9599-d3466c7375df None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:00.256485 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b1219-1cc6-462b-af85-542ad53d2095 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:00.301801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c51ea-88ed-4e89-98d0-151b125e23a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:00.302772 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c51ea-88ed-4e89-98d0-151b125e23a8 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:00.303681 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c51ea-88ed-4e89-98d0-151b125e23a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:00.304057 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c51ea-88ed-4e89-98d0-151b125e23a8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:00.320022 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966c51ea-88ed-4e89-98d0-151b125e23a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:00.327777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-966c51ea-88ed-4e89-98d0-151b125e23a8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 12:19:00.327777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:00.327777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:00.327777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:00.327777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:00.327777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:00.327777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:00.327777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:00.327777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:00.327777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:00.327777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:00.327777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:00.327777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:00.327777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:00.327777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:00.327777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:00.328969 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:00.328969 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:00.328969 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:00.328969 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:00.328969 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:00.328969 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:00.328969 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:00.328969 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:00.328969 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:00.328969 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:00.328969 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:00.328969 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:00.328969 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:00.328969 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:00.328969 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:00.328969 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:00.330033 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:00.330033 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:00.330033 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:00.330033 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:00.330033 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:00.330033 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:00.330033 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:00.330033 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:00.330033 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:00.330033 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:00.330033 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 12:19:00.330033 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:00.331105 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 172/350] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:19:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:00.332660 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f2cfa-8e4f-4f17-81d2-0e39aa40710e None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:00.332660 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f2cfa-8e4f-4f17-81d2-0e39aa40710e None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:00.332929 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f2cfa-8e4f-4f17-81d2-0e39aa40710e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:00.351301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-e44f2cfa-8e4f-4f17-81d2-0e39aa40710e None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:00.352613 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-e44f2cfa-8e4f-4f17-81d2-0e39aa40710e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:00.354551 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:00.358465 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:00.361534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97801c-ee8b-49ab-9599-d3466c7375df None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:00.408460 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386457b4-34f9-48b4-9c2a-e1f17a737cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:00.409029 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386457b4-34f9-48b4-9c2a-e1f17a737cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:00.409307 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386457b4-34f9-48b4-9c2a-e1f17a737cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:00.410002 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386457b4-34f9-48b4-9c2a-e1f17a737cc7 None admin] PATH_INFO: `/v3/projects/2bb553c7fe6e4284b56d30c136b7a041` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:00.410916 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386457b4-34f9-48b4-9c2a-e1f17a737cc7 None admin] RBAC: Authorizing `identity:get_project(project_id=2bb553c7fe6e4284b56d30c136b7a041)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:00.412408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386457b4-34f9-48b4-9c2a-e1f17a737cc7 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:00.418989 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 179/351] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:19:00 2022] GET /identity/v3/projects/2bb553c7fe6e4284b56d30c136b7a041 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:00.418989 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752787c3-a256-4158-8d6f-2e2ff0129716 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:00.418989 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752787c3-a256-4158-8d6f-2e2ff0129716 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:00.418989 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752787c3-a256-4158-8d6f-2e2ff0129716 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:00.418989 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 180/352] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:00.419390 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752787c3-a256-4158-8d6f-2e2ff0129716 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:00.442510 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 173/353] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:00.442877 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f2cfa-8e4f-4f17-81d2-0e39aa40710e None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:00.570304 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752787c3-a256-4158-8d6f-2e2ff0129716 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:00.591157 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f2cfa-8e4f-4f17-81d2-0e39aa40710e None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:00.616984 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab004b37-ae35-4763-872e-f2610f34903c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:00.617563 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab004b37-ae35-4763-872e-f2610f34903c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:00.617845 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab004b37-ae35-4763-872e-f2610f34903c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:00.618122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab004b37-ae35-4763-872e-f2610f34903c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:00.618959 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab004b37-ae35-4763-872e-f2610f34903c None admin] RBAC: Adding query filter params (group.id=b14634a2a38a44db8071dd00d49ded4a, role.id=2d4732dacad6444e86d7fee3514afefb, scope.project.id=2bb553c7fe6e4284b56d30c136b7a041) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:00.619275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab004b37-ae35-4763-872e-f2610f34903c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:00.620765 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab004b37-ae35-4763-872e-f2610f34903c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:00.626809 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 181/354] 10.222.0.16 () {56 vars in 1376 bytes} [Mon May 23 12:19:00 2022] GET /identity/v3/role_assignments?group.id=b14634a2a38a44db8071dd00d49ded4a&scope.project.id=2bb553c7fe6e4284b56d30c136b7a041&role.id=2d4732dacad6444e86d7fee3514afefb => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:00.627184 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab004b37-ae35-4763-872e-f2610f34903c None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:00.645864 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26edab-dffe-4947-a6d1-06f663c4018f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:00.647328 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26edab-dffe-4947-a6d1-06f663c4018f None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:00.647833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26edab-dffe-4947-a6d1-06f663c4018f None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:00.648385 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26edab-dffe-4947-a6d1-06f663c4018f None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:00.649395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26edab-dffe-4947-a6d1-06f663c4018f None admin] RBAC: Adding query filter params (name=member) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:00.649918 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26edab-dffe-4947-a6d1-06f663c4018f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:00.651787 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26edab-dffe-4947-a6d1-06f663c4018f None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:00.661974 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 174/355] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 12:19:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:00.663982 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:00.665832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a97c7d-3932-46ac-856f-4db3d1d8268a None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:00.666355 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a97c7d-3932-46ac-856f-4db3d1d8268a None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:00.666836 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a97c7d-3932-46ac-856f-4db3d1d8268a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:00.673194 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:00.688486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-a0a97c7d-3932-46ac-856f-4db3d1d8268a None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:00.690037 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-a0a97c7d-3932-46ac-856f-4db3d1d8268a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:00.767380 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab004b37-ae35-4763-872e-f2610f34903c None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:00.784878 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 175/356] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:00.785756 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a97c7d-3932-46ac-856f-4db3d1d8268a None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:00.816882 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10e459-28b8-4fcd-b156-1db74ea6647f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:00.818151 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10e459-28b8-4fcd-b156-1db74ea6647f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:00.818641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10e459-28b8-4fcd-b156-1db74ea6647f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:00.819673 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10e459-28b8-4fcd-b156-1db74ea6647f None admin] PATH_INFO: `/v3/groups/b14634a2a38a44db8071dd00d49ded4a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:00.821461 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10e459-28b8-4fcd-b156-1db74ea6647f None admin] RBAC: Authorizing `identity:get_group(group_id=b14634a2a38a44db8071dd00d49ded4a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:00.823178 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e10e459-28b8-4fcd-b156-1db74ea6647f None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:00.825477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 182/357] 10.222.0.16 () {56 vars in 1180 bytes} [Mon May 23 12:19:00 2022] GET /identity/v3/groups/b14634a2a38a44db8071dd00d49ded4a => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:00.826565 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10e459-28b8-4fcd-b156-1db74ea6647f None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:00.915998 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a97c7d-3932-46ac-856f-4db3d1d8268a None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:00.940096 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:00.947577 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e10e459-28b8-4fcd-b156-1db74ea6647f None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:00.959115 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:00.971080 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3336c4ae-8667-4e60-9af5-1f9d7dfd1d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:00.971892 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3336c4ae-8667-4e60-9af5-1f9d7dfd1d4d None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:00.972264 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3336c4ae-8667-4e60-9af5-1f9d7dfd1d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:00.972526 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3336c4ae-8667-4e60-9af5-1f9d7dfd1d4d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:00.995960 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3336c4ae-8667-4e60-9af5-1f9d7dfd1d4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:00.996213 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e4946-7a12-4be2-b81f-c62e9a2d9c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:00.996887 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e4946-7a12-4be2-b81f-c62e9a2d9c8a None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:00.997175 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e4946-7a12-4be2-b81f-c62e9a2d9c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:00.997458 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e4946-7a12-4be2-b81f-c62e9a2d9c8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:01.012434 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-3336c4ae-8667-4e60-9af5-1f9d7dfd1d4d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 12:19:01.012434 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:01.012434 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:01.012434 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:01.012434 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:01.012434 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:01.012434 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:01.012434 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:01.012434 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:01.012434 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:01.012434 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:01.012434 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:01.012434 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:01.012434 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:01.012434 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:01.012434 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:01.013568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:01.013568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:01.013568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:01.013568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:01.013568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:01.013568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:01.013568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:01.013568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:01.013568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:01.013568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:01.013568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:01.013568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:01.013568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:01.013568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:01.013568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:01.013568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:01.014568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:01.014568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:01.014568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:01.014568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:01.014568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:01.014568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:01.014568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:01.014568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:01.014568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:01.014568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:01.014568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 12:19:01.014568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:01.014568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 176/358] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:19:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:01.015246 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1232de8-269e-4ea0-b7a0-3b458fcd8578 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:01.015246 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1232de8-269e-4ea0-b7a0-3b458fcd8578 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:01.015451 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1232de8-269e-4ea0-b7a0-3b458fcd8578 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:01.018539 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 177/359] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:01.018539 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1232de8-269e-4ea0-b7a0-3b458fcd8578 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:01.091244 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e4946-7a12-4be2-b81f-c62e9a2d9c8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:01.092765 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e4946-7a12-4be2-b81f-c62e9a2d9c8a None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:01.160868 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1232de8-269e-4ea0-b7a0-3b458fcd8578 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:01.211402 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc8781-4d88-4653-a4a8-f909108ee358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:01.212009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:01.212136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc8781-4d88-4653-a4a8-f909108ee358 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:01.212683 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc8781-4d88-4653-a4a8-f909108ee358 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:01.212964 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc8781-4d88-4653-a4a8-f909108ee358 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:01.218622 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 183/360] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:19:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:19:01.219666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e4946-7a12-4be2-b81f-c62e9a2d9c8a None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:01.232821 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babc8781-4d88-4653-a4a8-f909108ee358 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:01.241636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-babc8781-4d88-4653-a4a8-f909108ee358 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 12:19:01.241636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:01.241636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:01.241636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:01.241636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:01.241636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:01.241636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:01.241636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:01.241636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:01.241636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:01.241636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:01.241636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:01.241636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:01.241636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:01.241636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:01.241636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:01.242574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:01.242574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:01.242574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:01.242574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:01.242574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:01.242574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:01.242574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:01.242574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:01.242574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:01.242574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:01.242574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:01.242574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:01.242574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:01.242574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:01.242574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:01.242574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:01.243401 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:01.243401 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:01.243401 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:01.243401 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:01.243401 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:01.243401 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:01.243401 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:01.243401 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:01.243401 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:01.243401 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:01.243401 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 12:19:01.243401 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:01.244113 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 178/361] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 12:19:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:01.244404 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc8781-4d88-4653-a4a8-f909108ee358 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:01.324356 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:01.360276 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e4946-7a12-4be2-b81f-c62e9a2d9c8a None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:01.393690 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc8781-4d88-4653-a4a8-f909108ee358 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:01.409146 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35521ec-fc08-4950-89b2-92623f67ad1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:01.410368 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35521ec-fc08-4950-89b2-92623f67ad1d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:01.410997 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35521ec-fc08-4950-89b2-92623f67ad1d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:01.411359 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35521ec-fc08-4950-89b2-92623f67ad1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:01.412325 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35521ec-fc08-4950-89b2-92623f67ad1d None admin] RBAC: Adding query filter params (group.id=b14634a2a38a44db8071dd00d49ded4a, role.id=504a45d998f0464893c90fa5429275a8, scope.project.id=2bb553c7fe6e4284b56d30c136b7a041) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:01.412830 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35521ec-fc08-4950-89b2-92623f67ad1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:01.414988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35521ec-fc08-4950-89b2-92623f67ad1d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:01.422927 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 184/362] 10.222.0.16 () {56 vars in 1376 bytes} [Mon May 23 12:19:01 2022] GET /identity/v3/role_assignments?group.id=b14634a2a38a44db8071dd00d49ded4a&scope.project.id=2bb553c7fe6e4284b56d30c136b7a041&role.id=504a45d998f0464893c90fa5429275a8 => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:01.424539 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245809c-04e0-42e4-b5de-45604fe5cd3d None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:01.424757 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245809c-04e0-42e4-b5de-45604fe5cd3d None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:01.425036 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245809c-04e0-42e4-b5de-45604fe5cd3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:01.444514 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e79dc-00e4-44eb-860f-72e7591e37c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:01.445615 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e79dc-00e4-44eb-860f-72e7591e37c2 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:01.446089 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e79dc-00e4-44eb-860f-72e7591e37c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:01.446587 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997e79dc-00e4-44eb-860f-72e7591e37c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:01.447546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e79dc-00e4-44eb-860f-72e7591e37c2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:01.448069 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e79dc-00e4-44eb-860f-72e7591e37c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:01.449753 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997e79dc-00e4-44eb-860f-72e7591e37c2 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:01.451015 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-5245809c-04e0-42e4-b5de-45604fe5cd3d None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:01.452514 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5245809c-04e0-42e4-b5de-45604fe5cd3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:01.461141 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 179/363] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 12:19:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:01.462410 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e79dc-00e4-44eb-860f-72e7591e37c2 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:01.540706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:01.556278 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 185/364] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:01.556994 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245809c-04e0-42e4-b5de-45604fe5cd3d None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:01.596162 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997e79dc-00e4-44eb-860f-72e7591e37c2 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:01.614503 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:01.645136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdceeadf-3d4b-4883-8992-127585602c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:01.646441 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdceeadf-3d4b-4883-8992-127585602c85 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:01.647502 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdceeadf-3d4b-4883-8992-127585602c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:01.647987 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdceeadf-3d4b-4883-8992-127585602c85 None admin] PATH_INFO: `/v3/projects/49c3998144de4918b868b834ea2918dc` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:01.649540 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdceeadf-3d4b-4883-8992-127585602c85 None admin] RBAC: Authorizing `identity:get_project(project_id=49c3998144de4918b868b834ea2918dc)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:01.651407 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdceeadf-3d4b-4883-8992-127585602c85 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:01.653920 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 180/365] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:19:01 2022] GET /identity/v3/projects/49c3998144de4918b868b834ea2918dc => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:01.655106 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdceeadf-3d4b-4883-8992-127585602c85 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:01.702123 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245809c-04e0-42e4-b5de-45604fe5cd3d None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:01.757993 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9e488-b721-4901-99b7-02902c908e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:01.758507 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9e488-b721-4901-99b7-02902c908e27 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:01.758791 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9e488-b721-4901-99b7-02902c908e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:01.759065 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9e488-b721-4901-99b7-02902c908e27 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:01.759825 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc9e488-b721-4901-99b7-02902c908e27 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:01.760121 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc9e488-b721-4901-99b7-02902c908e27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:01.761618 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc9e488-b721-4901-99b7-02902c908e27 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:01.768290 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdceeadf-3d4b-4883-8992-127585602c85 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:01.776584 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 186/366] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:01.777160 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9e488-b721-4901-99b7-02902c908e27 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:01.807660 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b69c72-0620-4f65-8e67-c56e3d3da880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:01.809128 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b69c72-0620-4f65-8e67-c56e3d3da880 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:01.809593 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b69c72-0620-4f65-8e67-c56e3d3da880 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:01.810062 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b69c72-0620-4f65-8e67-c56e3d3da880 None admin] PATH_INFO: `/v3/users/303fae1403d945889bb6dffdae1b7e52` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:01.811452 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b69c72-0620-4f65-8e67-c56e3d3da880 None admin] RBAC: Authorizing `identity:get_user(user_id=303fae1403d945889bb6dffdae1b7e52)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:01.813151 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b69c72-0620-4f65-8e67-c56e3d3da880 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:01.815220 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 181/367] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:19:01 2022] GET /identity/v3/users/303fae1403d945889bb6dffdae1b7e52 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:01.816227 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b69c72-0620-4f65-8e67-c56e3d3da880 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:01.843246 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:01.879683 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:01.922004 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9e488-b721-4901-99b7-02902c908e27 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:01.956436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b69c72-0620-4f65-8e67-c56e3d3da880 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:01.979740 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad0142-33f1-42f2-b685-040008c9ebf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:01.982009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad0142-33f1-42f2-b685-040008c9ebf2 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:01.982744 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad0142-33f1-42f2-b685-040008c9ebf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:01.983249 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad0142-33f1-42f2-b685-040008c9ebf2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:01.998709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df9a1e-398c-4030-8596-3685ceaf4f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:02.000034 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df9a1e-398c-4030-8596-3685ceaf4f1e None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:02.000469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df9a1e-398c-4030-8596-3685ceaf4f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:02.000635 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad0142-33f1-42f2-b685-040008c9ebf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:02.000934 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df9a1e-398c-4030-8596-3685ceaf4f1e None admin] PATH_INFO: `/v3/groups/b14634a2a38a44db8071dd00d49ded4a` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:02.002267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df9a1e-398c-4030-8596-3685ceaf4f1e None admin] RBAC: Authorizing `identity:get_group(group_id=b14634a2a38a44db8071dd00d49ded4a)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:02.003884 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df9a1e-398c-4030-8596-3685ceaf4f1e None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:02.005868 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 182/368] 10.222.0.16 () {56 vars in 1180 bytes} [Mon May 23 12:19:01 2022] GET /identity/v3/groups/b14634a2a38a44db8071dd00d49ded4a => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:02.010400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-59ad0142-33f1-42f2-b685-040008c9ebf2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:19:02.010400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:02.010400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:02.010400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:02.010400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:02.010400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:02.010400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:02.010400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:02.010400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:02.010400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:02.010400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:02.010400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:02.010400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:02.010400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:02.010400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:02.010400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:02.011295 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:02.011295 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:02.011295 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:02.011295 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:02.011295 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:02.011295 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:02.011295 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:02.011295 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:02.011295 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:02.011295 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:02.011295 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:02.011295 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:02.011295 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:02.011295 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:02.011295 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:02.011295 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:02.012093 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:02.012093 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:02.012093 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:02.012093 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:02.012093 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:02.012093 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:02.012093 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:02.012093 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:02.012093 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:02.012093 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:02.012093 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:19:02.012093 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:02.012748 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df9a1e-398c-4030-8596-3685ceaf4f1e None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:02.015068 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 187/369] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:19:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:02.016518 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad0142-33f1-42f2-b685-040008c9ebf2 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:02.168495 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad0142-33f1-42f2-b685-040008c9ebf2 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:02.168933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df9a1e-398c-4030-8596-3685ceaf4f1e None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:02.190612 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:02.195011 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:02.220702 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb96b78-9a56-4037-a9bf-ccfda6316156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:02.220890 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96379d-7e7d-4caf-85c0-102cc875add1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:02.221481 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96379d-7e7d-4caf-85c0-102cc875add1 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:02.221926 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb96b78-9a56-4037-a9bf-ccfda6316156 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:02.222101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96379d-7e7d-4caf-85c0-102cc875add1 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:02.222398 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb96b78-9a56-4037-a9bf-ccfda6316156 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:02.222671 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96379d-7e7d-4caf-85c0-102cc875add1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:02.222856 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb96b78-9a56-4037-a9bf-ccfda6316156 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:02.223789 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba96379d-7e7d-4caf-85c0-102cc875add1 None admin] RBAC: Adding query filter params (role.id=c2f1748f5cc24b8181c02f29cfea7aaf, user.id=303fae1403d945889bb6dffdae1b7e52, scope.system=all) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:02.224301 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba96379d-7e7d-4caf-85c0-102cc875add1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:02.227458 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba96379d-7e7d-4caf-85c0-102cc875add1 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:02.241145 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 188/370] 10.222.0.16 () {56 vars in 1308 bytes} [Mon May 23 12:19:02 2022] GET /identity/v3/role_assignments?user.id=303fae1403d945889bb6dffdae1b7e52&scope.system=all&role.id=c2f1748f5cc24b8181c02f29cfea7aaf => generated 510 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:02.249600 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96379d-7e7d-4caf-85c0-102cc875add1 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:02.318598 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb96b78-9a56-4037-a9bf-ccfda6316156 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:02.320764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb96b78-9a56-4037-a9bf-ccfda6316156 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:02.447514 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96379d-7e7d-4caf-85c0-102cc875add1 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:02.490470 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463bfa6a-b726-41da-9818-208f7a1f7837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:02.492287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463bfa6a-b726-41da-9818-208f7a1f7837 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:02.492781 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463bfa6a-b726-41da-9818-208f7a1f7837 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:02.493270 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463bfa6a-b726-41da-9818-208f7a1f7837 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:02.494345 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463bfa6a-b726-41da-9818-208f7a1f7837 None admin] RBAC: Adding query filter params (group.id=b14634a2a38a44db8071dd00d49ded4a, role.id=504a45d998f0464893c90fa5429275a8, scope.project.id=49c3998144de4918b868b834ea2918dc) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:02.494877 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463bfa6a-b726-41da-9818-208f7a1f7837 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:02.496630 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463bfa6a-b726-41da-9818-208f7a1f7837 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:02.504743 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 189/371] 10.222.0.16 () {56 vars in 1376 bytes} [Mon May 23 12:19:02 2022] GET /identity/v3/role_assignments?group.id=b14634a2a38a44db8071dd00d49ded4a&scope.project.id=49c3998144de4918b868b834ea2918dc&role.id=504a45d998f0464893c90fa5429275a8 => generated 260 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:02.505920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463bfa6a-b726-41da-9818-208f7a1f7837 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:02.507786 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 183/372] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:19:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:19:02.521649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb96b78-9a56-4037-a9bf-ccfda6316156 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:02.550293 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:02.573208 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:02.643986 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463bfa6a-b726-41da-9818-208f7a1f7837 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:02.682330 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2c508-5106-48dd-83be-53e92240134e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:02.683298 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2c508-5106-48dd-83be-53e92240134e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:02.683744 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2c508-5106-48dd-83be-53e92240134e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:02.684203 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2c508-5106-48dd-83be-53e92240134e None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:02.685114 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2c508-5106-48dd-83be-53e92240134e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:02.685570 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2c508-5106-48dd-83be-53e92240134e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:02.687198 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2c508-5106-48dd-83be-53e92240134e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:02.698300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 190/373] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:19:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:02.704199 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2c508-5106-48dd-83be-53e92240134e None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:02.712061 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb96b78-9a56-4037-a9bf-ccfda6316156 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:02.750544 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd623b46-4875-4492-a92d-82355cb1238e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:02.751423 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd623b46-4875-4492-a92d-82355cb1238e None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:02.751703 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd623b46-4875-4492-a92d-82355cb1238e None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:02.751978 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd623b46-4875-4492-a92d-82355cb1238e None admin] PATH_INFO: `/v3/projects/49c3998144de4918b868b834ea2918dc` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:02.754482 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd623b46-4875-4492-a92d-82355cb1238e None admin] RBAC: Authorizing `identity:get_project(project_id=49c3998144de4918b868b834ea2918dc)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:02.756080 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd623b46-4875-4492-a92d-82355cb1238e None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:02.758895 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 184/374] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:19:02 2022] GET /identity/v3/projects/49c3998144de4918b868b834ea2918dc => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:19:02.764497 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd623b46-4875-4492-a92d-82355cb1238e None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:02.816813 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:02.824404 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2c508-5106-48dd-83be-53e92240134e None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:02.887420 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021d029-6bc4-41bb-b75a-f77967d48d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:02.888497 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021d029-6bc4-41bb-b75a-f77967d48d8e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:02.889005 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021d029-6bc4-41bb-b75a-f77967d48d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:02.889502 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021d029-6bc4-41bb-b75a-f77967d48d8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:02.892758 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:02.916197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd623b46-4875-4492-a92d-82355cb1238e None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:02.955107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17952cd-55fc-482c-af61-6cfe5125b962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:02.956119 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17952cd-55fc-482c-af61-6cfe5125b962 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:02.956341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17952cd-55fc-482c-af61-6cfe5125b962 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:02.956605 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17952cd-55fc-482c-af61-6cfe5125b962 None admin] PATH_INFO: `/v3/groups/b14634a2a38a44db8071dd00d49ded4a` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:02.957780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17952cd-55fc-482c-af61-6cfe5125b962 None admin] RBAC: Authorizing `identity:get_group(group_id=b14634a2a38a44db8071dd00d49ded4a)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:02.959263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17952cd-55fc-482c-af61-6cfe5125b962 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:02.961714 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 185/375] 10.222.0.16 () {56 vars in 1180 bytes} [Mon May 23 12:19:02 2022] GET /identity/v3/groups/b14634a2a38a44db8071dd00d49ded4a => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:02.967424 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17952cd-55fc-482c-af61-6cfe5125b962 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:02.983163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e021d029-6bc4-41bb-b75a-f77967d48d8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:02.985036 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e021d029-6bc4-41bb-b75a-f77967d48d8e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:03.108542 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:03.112430 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 191/376] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:19:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 12:19:03.119134 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021d029-6bc4-41bb-b75a-f77967d48d8e None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:03.175337 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17952cd-55fc-482c-af61-6cfe5125b962 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:03.256575 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f176288-7f06-47b2-be6a-35d48ed5b504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:03.257712 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f176288-7f06-47b2-be6a-35d48ed5b504 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:03.259066 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f176288-7f06-47b2-be6a-35d48ed5b504 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:03.260676 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f176288-7f06-47b2-be6a-35d48ed5b504 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:03.261752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f176288-7f06-47b2-be6a-35d48ed5b504 None admin] RBAC: Adding query filter params (group.id=b14634a2a38a44db8071dd00d49ded4a, role.id=2d4732dacad6444e86d7fee3514afefb, scope.project.id=49c3998144de4918b868b834ea2918dc) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:03.264741 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f176288-7f06-47b2-be6a-35d48ed5b504 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:03.266535 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f176288-7f06-47b2-be6a-35d48ed5b504 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:03.279168 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 186/377] 10.222.0.16 () {56 vars in 1376 bytes} [Mon May 23 12:19:02 2022] GET /identity/v3/role_assignments?group.id=b14634a2a38a44db8071dd00d49ded4a&scope.project.id=49c3998144de4918b868b834ea2918dc&role.id=2d4732dacad6444e86d7fee3514afefb => generated 260 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:19:03.313514 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021d029-6bc4-41bb-b75a-f77967d48d8e None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:03.336498 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cc796-5f29-42f1-9563-cfd21de7a9fb None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:03.336830 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cc796-5f29-42f1-9563-cfd21de7a9fb None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:03.337059 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cc796-5f29-42f1-9563-cfd21de7a9fb None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:03.338663 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 187/378] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:19:03.344799 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368e50a-711d-473d-91e6-60d693a62582 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:03.345153 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368e50a-711d-473d-91e6-60d693a62582 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:03.345439 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368e50a-711d-473d-91e6-60d693a62582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:03.356498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361147e-b443-40c1-8028-db6ccd48bd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:03.357757 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361147e-b443-40c1-8028-db6ccd48bd69 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:03.358271 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361147e-b443-40c1-8028-db6ccd48bd69 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:03.358824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361147e-b443-40c1-8028-db6ccd48bd69 None admin] PATH_INFO: `/v3/projects/c631b5a3911e4647a05ad6138d2bc618` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:03.360401 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b361147e-b443-40c1-8028-db6ccd48bd69 None admin] RBAC: Authorizing `identity:get_project(project_id=c631b5a3911e4647a05ad6138d2bc618)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:03.362146 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b361147e-b443-40c1-8028-db6ccd48bd69 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:03.364640 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 192/379] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:19:03 2022] GET /identity/v3/projects/c631b5a3911e4647a05ad6138d2bc618 => generated 344 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:03.370633 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361147e-b443-40c1-8028-db6ccd48bd69 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:03.372102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-a368e50a-711d-473d-91e6-60d693a62582 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:03.373329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-a368e50a-711d-473d-91e6-60d693a62582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:03.393975 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:03.446916 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:03.464154 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 188/380] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 12:19:03.489739 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361147e-b443-40c1-8028-db6ccd48bd69 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:03.494484 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368e50a-711d-473d-91e6-60d693a62582 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:03.530233 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19643e37-8027-48a5-8aba-1ec9409f6f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:03.531481 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19643e37-8027-48a5-8aba-1ec9409f6f52 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:03.531967 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19643e37-8027-48a5-8aba-1ec9409f6f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:03.532467 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19643e37-8027-48a5-8aba-1ec9409f6f52 None admin] PATH_INFO: `/v3/groups/2b059d0198ff417d94197b72d0594da1` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:03.537965 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19643e37-8027-48a5-8aba-1ec9409f6f52 None admin] RBAC: Authorizing `identity:get_group(group_id=2b059d0198ff417d94197b72d0594da1)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:03.539621 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19643e37-8027-48a5-8aba-1ec9409f6f52 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:03.541838 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 193/381] 10.222.0.16 () {56 vars in 1180 bytes} [Mon May 23 12:19:03 2022] GET /identity/v3/groups/2b059d0198ff417d94197b72d0594da1 => generated 229 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:19:03.547874 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19643e37-8027-48a5-8aba-1ec9409f6f52 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:03.613208 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368e50a-711d-473d-91e6-60d693a62582 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:03.653907 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba0002-8d88-49a1-a1fb-8bc7ec5c62aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:03.654717 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba0002-8d88-49a1-a1fb-8bc7ec5c62aa None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:03.655000 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba0002-8d88-49a1-a1fb-8bc7ec5c62aa None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:03.655275 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba0002-8d88-49a1-a1fb-8bc7ec5c62aa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:03.664665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19643e37-8027-48a5-8aba-1ec9409f6f52 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:03.670368 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba0002-8d88-49a1-a1fb-8bc7ec5c62aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:03.679005 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-2cba0002-8d88-49a1-a1fb-8bc7ec5c62aa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 12:19:03.679005 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:03.679005 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:03.679005 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:03.679005 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:03.679005 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:03.679005 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:03.679005 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:03.679005 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:03.679005 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:03.679005 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:03.679005 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:03.679005 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:03.679005 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:03.679005 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:03.679005 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:03.679983 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:03.679983 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:03.679983 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:03.679983 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:03.679983 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:03.679983 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:03.679983 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:03.679983 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:03.679983 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:03.679983 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:03.679983 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:03.679983 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:03.679983 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:03.679983 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:03.679983 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:03.679983 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:03.680851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:03.680851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:03.680851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:03.680851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:03.680851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:03.680851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:03.680851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:03.680851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:03.680851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:03.680851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:03.680851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 12:19:03.680851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:03.680851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 189/382] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 12:19:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:19:03.689642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba0002-8d88-49a1-a1fb-8bc7ec5c62aa None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:03.703002 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:03.726667 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:03.748322 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fec12a-2d54-4672-8cc2-a59e0b8b012b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:03.748999 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fec12a-2d54-4672-8cc2-a59e0b8b012b None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:03.749290 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fec12a-2d54-4672-8cc2-a59e0b8b012b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:03.749571 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fec12a-2d54-4672-8cc2-a59e0b8b012b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:03.750422 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fec12a-2d54-4672-8cc2-a59e0b8b012b None admin] RBAC: Adding query filter params (group.id=2b059d0198ff417d94197b72d0594da1, role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8, scope.project.id=c631b5a3911e4647a05ad6138d2bc618) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:03.750737 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fec12a-2d54-4672-8cc2-a59e0b8b012b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:03.752191 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fec12a-2d54-4672-8cc2-a59e0b8b012b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:03.758987 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 194/383] 10.222.0.16 () {56 vars in 1376 bytes} [Mon May 23 12:19:03 2022] GET /identity/v3/role_assignments?group.id=2b059d0198ff417d94197b72d0594da1&scope.project.id=c631b5a3911e4647a05ad6138d2bc618&role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:19:03.838458 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba0002-8d88-49a1-a1fb-8bc7ec5c62aa None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:03.893029 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bbfec2-1b66-40d2-aac0-f6adbdaaa0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:03.893628 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbfec2-1b66-40d2-aac0-f6adbdaaa0fc None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:03.893913 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbfec2-1b66-40d2-aac0-f6adbdaaa0fc None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:03.894191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbfec2-1b66-40d2-aac0-f6adbdaaa0fc None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:03.894941 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bbfec2-1b66-40d2-aac0-f6adbdaaa0fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:03.895251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bbfec2-1b66-40d2-aac0-f6adbdaaa0fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:03.896669 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bbfec2-1b66-40d2-aac0-f6adbdaaa0fc None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:03.911931 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 190/384] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:03.918188 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fec12a-2d54-4672-8cc2-a59e0b8b012b None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:04.080910 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fec12a-2d54-4672-8cc2-a59e0b8b012b None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:04.121887 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c214bcb-9dee-4e29-a26d-1c38cf32cbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:04.122751 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c214bcb-9dee-4e29-a26d-1c38cf32cbd5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:04.123028 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c214bcb-9dee-4e29-a26d-1c38cf32cbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:04.123303 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c214bcb-9dee-4e29-a26d-1c38cf32cbd5 None admin] PATH_INFO: `/v3/groups/b14634a2a38a44db8071dd00d49ded4a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:04.124574 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c214bcb-9dee-4e29-a26d-1c38cf32cbd5 None admin] RBAC: Authorizing `identity:get_group(group_id=b14634a2a38a44db8071dd00d49ded4a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:04.126066 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c214bcb-9dee-4e29-a26d-1c38cf32cbd5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:04.127966 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 195/385] 10.222.0.16 () {56 vars in 1180 bytes} [Mon May 23 12:19:03 2022] GET /identity/v3/groups/b14634a2a38a44db8071dd00d49ded4a => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:04.133991 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bbfec2-1b66-40d2-aac0-f6adbdaaa0fc None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:04.217669 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:04.258168 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bbfec2-1b66-40d2-aac0-f6adbdaaa0fc None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:04.307287 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f123b537-9932-4975-a17e-ad9021cc6a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:04.308190 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123b537-9932-4975-a17e-ad9021cc6a84 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:04.308482 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123b537-9932-4975-a17e-ad9021cc6a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:04.308802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123b537-9932-4975-a17e-ad9021cc6a84 None admin] PATH_INFO: `/v3/projects/2bb553c7fe6e4284b56d30c136b7a041` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:04.310044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f123b537-9932-4975-a17e-ad9021cc6a84 None admin] RBAC: Authorizing `identity:get_project(project_id=2bb553c7fe6e4284b56d30c136b7a041)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:04.311517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f123b537-9932-4975-a17e-ad9021cc6a84 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:04.313562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 191/386] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:19:04 2022] GET /identity/v3/projects/2bb553c7fe6e4284b56d30c136b7a041 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:04.319866 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c214bcb-9dee-4e29-a26d-1c38cf32cbd5 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:04.429053 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:04.493579 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c214bcb-9dee-4e29-a26d-1c38cf32cbd5 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:04.542573 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be974879-8e65-4d8a-b4d6-7fe3fe9938ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:04.543818 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be974879-8e65-4d8a-b4d6-7fe3fe9938ed None admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:04.544340 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be974879-8e65-4d8a-b4d6-7fe3fe9938ed None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:04.544826 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be974879-8e65-4d8a-b4d6-7fe3fe9938ed None admin] PATH_INFO: `/v3/projects/2bb553c7fe6e4284b56d30c136b7a041/groups/b14634a2a38a44db8071dd00d49ded4a/roles/2d4732dacad6444e86d7fee3514afefb` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:04.548914 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be974879-8e65-4d8a-b4d6-7fe3fe9938ed None admin] RBAC: Authorizing `identity:create_grant(project_id=2bb553c7fe6e4284b56d30c136b7a041, group_id=b14634a2a38a44db8071dd00d49ded4a, role_id=2d4732dacad6444e86d7fee3514afefb)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:04.550549 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be974879-8e65-4d8a-b4d6-7fe3fe9938ed None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:04.578232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 196/387] 10.222.0.16 () {58 vars in 1442 bytes} [Mon May 23 12:19:04 2022] PUT /identity/v3/projects/2bb553c7fe6e4284b56d30c136b7a041/groups/b14634a2a38a44db8071dd00d49ded4a/roles/2d4732dacad6444e86d7fee3514afefb => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:19:04.610965 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade752e-901d-4a1e-9aa7-e484fa615f7b None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:04.611263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade752e-901d-4a1e-9aa7-e484fa615f7b None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:04.611754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade752e-901d-4a1e-9aa7-e484fa615f7b None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:04.613448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 192/388] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:04.619410 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024dbae-9bc4-43a1-bc9d-4d9e56f560a3 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:04.619713 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024dbae-9bc4-43a1-bc9d-4d9e56f560a3 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:04.619985 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024dbae-9bc4-43a1-bc9d-4d9e56f560a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:04.642091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-d024dbae-9bc4-43a1-bc9d-4d9e56f560a3 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:04.643293 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d024dbae-9bc4-43a1-bc9d-4d9e56f560a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:04.722622 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 197/389] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:04.746657 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade752e-901d-4a1e-9aa7-e484fa615f7b None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:04.895009 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade752e-901d-4a1e-9aa7-e484fa615f7b None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:04.938613 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f90cf-e5fb-4113-b55e-bf0d744fc851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:04.939420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f90cf-e5fb-4113-b55e-bf0d744fc851 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:04.939710 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f90cf-e5fb-4113-b55e-bf0d744fc851 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:04.939990 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f90cf-e5fb-4113-b55e-bf0d744fc851 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:04.949395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:04.955154 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f90cf-e5fb-4113-b55e-bf0d744fc851 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:04.965505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-f08f90cf-e5fb-4113-b55e-bf0d744fc851 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 12:19:04.965505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:04.965505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:04.965505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:04.965505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:04.965505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:04.965505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:04.965505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:04.965505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:04.965505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:04.965505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:04.965505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:04.965505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:04.965505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:04.965505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:04.965505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:04.966423 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:04.966423 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:04.966423 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:04.966423 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:04.966423 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:04.966423 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:04.966423 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:04.966423 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:04.966423 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:04.966423 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:04.966423 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:04.966423 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:04.966423 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:04.966423 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:04.966423 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:04.966423 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:04.967327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:04.967327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:04.967327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:04.967327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:04.967327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:04.967327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:04.967327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:04.967327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:04.967327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:04.967327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:04.967327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 12:19:04.967327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:04.967327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 193/390] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:19:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:04.974313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024dbae-9bc4-43a1-bc9d-4d9e56f560a3 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:05.076525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:05.147907 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024dbae-9bc4-43a1-bc9d-4d9e56f560a3 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:05.200881 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458f0c0-beda-4b52-8966-e21ccf448331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:05.201552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458f0c0-beda-4b52-8966-e21ccf448331 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:05.201834 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458f0c0-beda-4b52-8966-e21ccf448331 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:05.202110 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458f0c0-beda-4b52-8966-e21ccf448331 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:05.202857 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458f0c0-beda-4b52-8966-e21ccf448331 None admin] RBAC: Adding query filter params (name=member) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:05.203166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458f0c0-beda-4b52-8966-e21ccf448331 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:05.204563 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458f0c0-beda-4b52-8966-e21ccf448331 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:05.215761 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 198/391] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 12:19:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:05.221320 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f90cf-e5fb-4113-b55e-bf0d744fc851 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:05.343707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f90cf-e5fb-4113-b55e-bf0d744fc851 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:05.391677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b133d-c4f6-4f32-a644-dd4cb49973f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:05.392570 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b133d-c4f6-4f32-a644-dd4cb49973f3 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:05.392852 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b133d-c4f6-4f32-a644-dd4cb49973f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:05.393126 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b133d-c4f6-4f32-a644-dd4cb49973f3 None admin] PATH_INFO: `/v3/groups/b14634a2a38a44db8071dd00d49ded4a` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:05.397047 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653b133d-c4f6-4f32-a644-dd4cb49973f3 None admin] RBAC: Authorizing `identity:get_group(group_id=b14634a2a38a44db8071dd00d49ded4a)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:05.398703 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653b133d-c4f6-4f32-a644-dd4cb49973f3 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:05.400377 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 194/392] 10.222.0.16 () {56 vars in 1180 bytes} [Mon May 23 12:19:05 2022] GET /identity/v3/groups/b14634a2a38a44db8071dd00d49ded4a => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:05.405965 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458f0c0-beda-4b52-8966-e21ccf448331 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:05.544515 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459cf85-fb16-4465-a5f9-54208655e550 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:05.544861 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459cf85-fb16-4465-a5f9-54208655e550 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:05.545091 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459cf85-fb16-4465-a5f9-54208655e550 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:05.550184 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 195/393] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:05.557558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cba42-40ce-40b8-8b0f-8e2c3d37ca3f None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:05.557847 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cba42-40ce-40b8-8b0f-8e2c3d37ca3f None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:05.558129 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cba42-40ce-40b8-8b0f-8e2c3d37ca3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:05.558876 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458f0c0-beda-4b52-8966-e21ccf448331 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:05.573632 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:05.590647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-826cba42-40ce-40b8-8b0f-8e2c3d37ca3f None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:05.590647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-826cba42-40ce-40b8-8b0f-8e2c3d37ca3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:05.613288 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e3f16-b3e6-42ce-829a-73a0b21a4d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:05.614169 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e3f16-b3e6-42ce-829a-73a0b21a4d73 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:05.614452 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e3f16-b3e6-42ce-829a-73a0b21a4d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:05.614721 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e3f16-b3e6-42ce-829a-73a0b21a4d73 None admin] PATH_INFO: `/v3/projects/2bb553c7fe6e4284b56d30c136b7a041` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:05.617897 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e3f16-b3e6-42ce-829a-73a0b21a4d73 None admin] RBAC: Authorizing `identity:get_project(project_id=2bb553c7fe6e4284b56d30c136b7a041)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:05.619355 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e3f16-b3e6-42ce-829a-73a0b21a4d73 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:05.621083 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 199/394] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:19:05 2022] GET /identity/v3/projects/2bb553c7fe6e4284b56d30c136b7a041 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:05.628468 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e3f16-b3e6-42ce-829a-73a0b21a4d73 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:05.706240 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:05.727641 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 196/395] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:05.762873 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cba42-40ce-40b8-8b0f-8e2c3d37ca3f None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:05.767515 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e3f16-b3e6-42ce-829a-73a0b21a4d73 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:05.803407 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ed629-b864-4fee-ba82-e3b1f78f48b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:05.804509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ed629-b864-4fee-ba82-e3b1f78f48b3 None admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:05.804807 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ed629-b864-4fee-ba82-e3b1f78f48b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:05.805096 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6ed629-b864-4fee-ba82-e3b1f78f48b3 None admin] PATH_INFO: `/v3/projects/2bb553c7fe6e4284b56d30c136b7a041/groups/b14634a2a38a44db8071dd00d49ded4a/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:05.807133 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ed629-b864-4fee-ba82-e3b1f78f48b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=2bb553c7fe6e4284b56d30c136b7a041, group_id=b14634a2a38a44db8071dd00d49ded4a, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:05.808616 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6ed629-b864-4fee-ba82-e3b1f78f48b3 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:05.830651 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 200/396] 10.222.0.16 () {58 vars in 1442 bytes} [Mon May 23 12:19:05 2022] PUT /identity/v3/projects/2bb553c7fe6e4284b56d30c136b7a041/groups/b14634a2a38a44db8071dd00d49ded4a/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:19:05.916831 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cba42-40ce-40b8-8b0f-8e2c3d37ca3f None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:05.955384 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c860105-cbfb-4daa-806e-5397a2513e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:05.956210 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c860105-cbfb-4daa-806e-5397a2513e70 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:05.957602 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c860105-cbfb-4daa-806e-5397a2513e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:05.957602 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c860105-cbfb-4daa-806e-5397a2513e70 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:05.970037 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c860105-cbfb-4daa-806e-5397a2513e70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:05.979021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-2c860105-cbfb-4daa-806e-5397a2513e70 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 12:19:05.979021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:05.979021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:05.979021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:05.979021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:05.979021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:05.979021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:05.979021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:05.979021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:05.979021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:05.979021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:05.979021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:05.979021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:05.979021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:05.979021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:05.979021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:05.980216 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:05.980216 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:05.980216 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:05.980216 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:05.980216 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:05.980216 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:05.980216 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:05.980216 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:05.980216 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:05.980216 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:05.980216 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:05.980216 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:05.980216 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:05.980216 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:05.980216 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:05.980216 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:05.981231 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:05.981231 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:05.981231 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:05.981231 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:05.981231 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:05.981231 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:05.981231 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:05.981231 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:05.981231 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:05.981231 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:05.981231 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 12:19:05.981231 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:05.983743 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 197/397] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:19:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:19:05.989779 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ed629-b864-4fee-ba82-e3b1f78f48b3 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:06.052592 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:06.129160 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6ed629-b864-4fee-ba82-e3b1f78f48b3 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:06.167038 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc8ae98-bb28-4cf5-9044-2952136be881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:06.167639 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8ae98-bb28-4cf5-9044-2952136be881 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:06.167926 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8ae98-bb28-4cf5-9044-2952136be881 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:06.168230 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8ae98-bb28-4cf5-9044-2952136be881 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:06.168991 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc8ae98-bb28-4cf5-9044-2952136be881 None admin] RBAC: Adding query filter params (name=member) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:06.169307 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc8ae98-bb28-4cf5-9044-2952136be881 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:06.170705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc8ae98-bb28-4cf5-9044-2952136be881 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:06.180022 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 201/398] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 12:19:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:06.185658 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c860105-cbfb-4daa-806e-5397a2513e70 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:06.251153 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:06.259758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171247b-a3c7-4c2d-9cd3-efeb456928bf None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:06.260049 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171247b-a3c7-4c2d-9cd3-efeb456928bf None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:06.260354 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171247b-a3c7-4c2d-9cd3-efeb456928bf None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:06.261958 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 202/399] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:19:06.267756 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc947d61-5669-4fe0-b47c-357521e85d8c None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:06.268042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc947d61-5669-4fe0-b47c-357521e85d8c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:06.268405 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc947d61-5669-4fe0-b47c-357521e85d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:06.293162 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-dc947d61-5669-4fe0-b47c-357521e85d8c None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:06.294348 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-dc947d61-5669-4fe0-b47c-357521e85d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:06.303974 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c860105-cbfb-4daa-806e-5397a2513e70 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:06.345205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f337b-1a0d-4351-b184-fbde0488eab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:06.346092 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f337b-1a0d-4351-b184-fbde0488eab2 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:06.346519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f337b-1a0d-4351-b184-fbde0488eab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:06.346814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f337b-1a0d-4351-b184-fbde0488eab2 None admin] PATH_INFO: `/v3/groups/b14634a2a38a44db8071dd00d49ded4a` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:06.348246 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85f337b-1a0d-4351-b184-fbde0488eab2 None admin] RBAC: Authorizing `identity:get_group(group_id=b14634a2a38a44db8071dd00d49ded4a)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:06.350869 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85f337b-1a0d-4351-b184-fbde0488eab2 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:06.353158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 198/400] 10.222.0.16 () {56 vars in 1180 bytes} [Mon May 23 12:19:06 2022] GET /identity/v3/groups/b14634a2a38a44db8071dd00d49ded4a => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:06.358553 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f337b-1a0d-4351-b184-fbde0488eab2 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:06.470505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f337b-1a0d-4351-b184-fbde0488eab2 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:06.472366 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 203/401] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:06.498984 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:06.507980 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f0efb-f512-4f95-8428-32f98d222990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:06.508886 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f0efb-f512-4f95-8428-32f98d222990 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:06.509179 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f0efb-f512-4f95-8428-32f98d222990 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:06.509307 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc947d61-5669-4fe0-b47c-357521e85d8c None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:06.509472 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f0efb-f512-4f95-8428-32f98d222990 None admin] PATH_INFO: `/v3/projects/49c3998144de4918b868b834ea2918dc` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:06.510687 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f0efb-f512-4f95-8428-32f98d222990 None admin] RBAC: Authorizing `identity:get_project(project_id=49c3998144de4918b868b834ea2918dc)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:06.512703 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f0efb-f512-4f95-8428-32f98d222990 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:06.519490 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 199/402] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:19:06 2022] GET /identity/v3/projects/49c3998144de4918b868b834ea2918dc => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:19:06.521406 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f0efb-f512-4f95-8428-32f98d222990 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:06.572677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:06.680732 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc947d61-5669-4fe0-b47c-357521e85d8c None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:06.719181 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f0efb-f512-4f95-8428-32f98d222990 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:06.722804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7690bc06-c4c1-4dd3-9633-e71a6597745c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:06.723653 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690bc06-c4c1-4dd3-9633-e71a6597745c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:06.724035 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690bc06-c4c1-4dd3-9633-e71a6597745c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:06.724433 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690bc06-c4c1-4dd3-9633-e71a6597745c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:06.735604 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7690bc06-c4c1-4dd3-9633-e71a6597745c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:06.744408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-7690bc06-c4c1-4dd3-9633-e71a6597745c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 12:19:06.744408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:06.744408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:06.744408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:06.744408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:06.744408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:06.744408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:06.744408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:06.744408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:06.744408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:06.744408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:06.744408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:06.744408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:06.744408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:06.744408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:06.744408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:06.745521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:06.745521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:06.745521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:06.745521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:06.745521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:06.745521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:06.745521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:06.745521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:06.745521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:06.745521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:06.745521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:06.745521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:06.745521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:06.745521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:06.745521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:06.745521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:06.746448 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:06.746448 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:06.746448 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:06.746448 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:06.746448 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:06.746448 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:06.746448 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:06.746448 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:06.746448 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:06.746448 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:06.746448 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 12:19:06.746448 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:06.747060 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 204/403] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 12:19:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:06.748364 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5049481-fcc6-4d69-b6ce-5e826bd25e62 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:06.749601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5049481-fcc6-4d69-b6ce-5e826bd25e62 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:06.749601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5049481-fcc6-4d69-b6ce-5e826bd25e62 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:06.750704 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 205/404] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:06.757739 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8342dc-574d-42b7-8494-fbe0324c7243 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:06.758027 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8342dc-574d-42b7-8494-fbe0324c7243 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:06.758308 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8342dc-574d-42b7-8494-fbe0324c7243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:06.763609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f151fa-b74b-4bc8-9df6-4382d232b94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:06.764760 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f151fa-b74b-4bc8-9df6-4382d232b94d None admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:06.765051 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f151fa-b74b-4bc8-9df6-4382d232b94d None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:06.765355 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f151fa-b74b-4bc8-9df6-4382d232b94d None admin] PATH_INFO: `/v3/projects/49c3998144de4918b868b834ea2918dc/groups/b14634a2a38a44db8071dd00d49ded4a/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:06.767604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f151fa-b74b-4bc8-9df6-4382d232b94d None admin] RBAC: Authorizing `identity:create_grant(project_id=49c3998144de4918b868b834ea2918dc, group_id=b14634a2a38a44db8071dd00d49ded4a, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:06.769089 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f151fa-b74b-4bc8-9df6-4382d232b94d None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:06.777465 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-bd8342dc-574d-42b7-8494-fbe0324c7243 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:06.778801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-bd8342dc-574d-42b7-8494-fbe0324c7243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:06.788133 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 200/405] 10.222.0.16 () {58 vars in 1442 bytes} [Mon May 23 12:19:06 2022] PUT /identity/v3/projects/49c3998144de4918b868b834ea2918dc/groups/b14634a2a38a44db8071dd00d49ded4a/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:19:06.789407 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f151fa-b74b-4bc8-9df6-4382d232b94d None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:06.849179 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:06.905358 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f151fa-b74b-4bc8-9df6-4382d232b94d None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:06.922103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 206/406] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:06.943100 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2898927-1318-4ca8-b942-fcf3a82781f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:06.944571 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2898927-1318-4ca8-b942-fcf3a82781f5 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:06.944962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2898927-1318-4ca8-b942-fcf3a82781f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:06.945539 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2898927-1318-4ca8-b942-fcf3a82781f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:06.946423 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2898927-1318-4ca8-b942-fcf3a82781f5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:06.946840 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2898927-1318-4ca8-b942-fcf3a82781f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:06.948373 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2898927-1318-4ca8-b942-fcf3a82781f5 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:06.962445 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 201/407] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:06.963255 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8342dc-574d-42b7-8494-fbe0324c7243 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:06.973405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2898927-1318-4ca8-b942-fcf3a82781f5 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:07.008554 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:07.082468 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8342dc-574d-42b7-8494-fbe0324c7243 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:07.120384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7942dcb-7830-481d-ad6f-c3f87e7e450b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:07.121350 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7942dcb-7830-481d-ad6f-c3f87e7e450b None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:07.121548 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7942dcb-7830-481d-ad6f-c3f87e7e450b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:07.121833 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7942dcb-7830-481d-ad6f-c3f87e7e450b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:07.134698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7942dcb-7830-481d-ad6f-c3f87e7e450b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:07.142656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-d7942dcb-7830-481d-ad6f-c3f87e7e450b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:19:07.142656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:07.142656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:07.142656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:07.142656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:07.142656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:07.142656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:07.142656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:07.142656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:07.142656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:07.142656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:07.142656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:07.142656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:07.142656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:07.142656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:07.142656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:07.143554 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:07.143554 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:07.143554 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:07.143554 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:07.143554 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:07.143554 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:07.143554 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:07.143554 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:07.143554 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:07.143554 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:07.143554 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:07.143554 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:07.143554 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:07.143554 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:07.143554 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:07.143554 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:07.144399 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:07.144399 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:07.144399 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:07.144399 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:07.144399 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:07.144399 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:07.144399 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:07.144399 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:07.144399 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:07.144399 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:07.144399 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:19:07.144399 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:07.144399 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 207/408] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:19:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:07.145686 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2898927-1318-4ca8-b942-fcf3a82781f5 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:07.150595 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7942dcb-7830-481d-ad6f-c3f87e7e450b None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:07.162413 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:07.190547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c635500-ddd9-4501-b4be-0ac1d7c26a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:07.194176 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c635500-ddd9-4501-b4be-0ac1d7c26a5f None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:07.194176 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c635500-ddd9-4501-b4be-0ac1d7c26a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:07.194176 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c635500-ddd9-4501-b4be-0ac1d7c26a5f None admin] PATH_INFO: `/v3/groups/b14634a2a38a44db8071dd00d49ded4a` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:07.195432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c635500-ddd9-4501-b4be-0ac1d7c26a5f None admin] RBAC: Authorizing `identity:get_group(group_id=b14634a2a38a44db8071dd00d49ded4a)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:07.197019 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c635500-ddd9-4501-b4be-0ac1d7c26a5f None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:07.199092 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 202/409] 10.222.0.16 () {56 vars in 1180 bytes} [Mon May 23 12:19:06 2022] GET /identity/v3/groups/b14634a2a38a44db8071dd00d49ded4a => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:07.204580 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c635500-ddd9-4501-b4be-0ac1d7c26a5f None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:07.255955 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:07.274021 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7942dcb-7830-481d-ad6f-c3f87e7e450b None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:07.347516 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fffc1-0992-43dd-a2b9-e2fcca543836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:07.348213 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fffc1-0992-43dd-a2b9-e2fcca543836 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:07.348506 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fffc1-0992-43dd-a2b9-e2fcca543836 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:07.348788 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840fffc1-0992-43dd-a2b9-e2fcca543836 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:07.349543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fffc1-0992-43dd-a2b9-e2fcca543836 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:07.350424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fffc1-0992-43dd-a2b9-e2fcca543836 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:07.352107 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840fffc1-0992-43dd-a2b9-e2fcca543836 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:07.363995 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 208/410] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:19:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:07.371116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c635500-ddd9-4501-b4be-0ac1d7c26a5f None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:07.373157 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fffc1-0992-43dd-a2b9-e2fcca543836 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:07.409648 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab03281-af37-48f2-b42d-b945b5775b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:07.411452 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab03281-af37-48f2-b42d-b945b5775b25 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:07.411974 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab03281-af37-48f2-b42d-b945b5775b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:07.412995 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab03281-af37-48f2-b42d-b945b5775b25 None admin] PATH_INFO: `/v3/projects/49c3998144de4918b868b834ea2918dc` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:07.414576 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab03281-af37-48f2-b42d-b945b5775b25 None admin] RBAC: Authorizing `identity:get_project(project_id=49c3998144de4918b868b834ea2918dc)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:07.416411 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab03281-af37-48f2-b42d-b945b5775b25 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:07.418613 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 203/411] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:19:07 2022] GET /identity/v3/projects/49c3998144de4918b868b834ea2918dc => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:19:07.423770 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab03281-af37-48f2-b42d-b945b5775b25 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:07.484449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:07.561389 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab03281-af37-48f2-b42d-b945b5775b25 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:07.576486 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840fffc1-0992-43dd-a2b9-e2fcca543836 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:07.611004 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afc8f26-a906-4836-bfa8-708ae5da556c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:07.612278 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc8f26-a906-4836-bfa8-708ae5da556c None admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:07.612837 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc8f26-a906-4836-bfa8-708ae5da556c None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:07.613307 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc8f26-a906-4836-bfa8-708ae5da556c None admin] PATH_INFO: `/v3/projects/49c3998144de4918b868b834ea2918dc/groups/b14634a2a38a44db8071dd00d49ded4a/roles/2d4732dacad6444e86d7fee3514afefb` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:07.614062 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c513d0-5c02-4248-815d-ed24ea1821a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:07.614881 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c513d0-5c02-4248-815d-ed24ea1821a2 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:07.615172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c513d0-5c02-4248-815d-ed24ea1821a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:07.615593 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c513d0-5c02-4248-815d-ed24ea1821a2 None admin] PATH_INFO: `/v3/groups/2b059d0198ff417d94197b72d0594da1` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:07.616873 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c513d0-5c02-4248-815d-ed24ea1821a2 None admin] RBAC: Authorizing `identity:get_group(group_id=2b059d0198ff417d94197b72d0594da1)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:07.616996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afc8f26-a906-4836-bfa8-708ae5da556c None admin] RBAC: Authorizing `identity:create_grant(project_id=49c3998144de4918b868b834ea2918dc, group_id=b14634a2a38a44db8071dd00d49ded4a, role_id=2d4732dacad6444e86d7fee3514afefb)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:07.618303 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c513d0-5c02-4248-815d-ed24ea1821a2 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:07.618843 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afc8f26-a906-4836-bfa8-708ae5da556c None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:07.624095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 209/412] 10.222.0.16 () {56 vars in 1180 bytes} [Mon May 23 12:19:07 2022] GET /identity/v3/groups/2b059d0198ff417d94197b72d0594da1 => generated 229 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:07.624095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0359610-9482-490e-901b-af80dfc4254d None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:07.624095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0359610-9482-490e-901b-af80dfc4254d None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:07.624095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0359610-9482-490e-901b-af80dfc4254d None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:07.624095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 210/413] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:07.626592 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0359610-9482-490e-901b-af80dfc4254d None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:07.646638 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 204/414] 10.222.0.16 () {58 vars in 1442 bytes} [Mon May 23 12:19:07 2022] PUT /identity/v3/projects/49c3998144de4918b868b834ea2918dc/groups/b14634a2a38a44db8071dd00d49ded4a/roles/2d4732dacad6444e86d7fee3514afefb => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:19:07.653619 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e53aa-c67c-4e32-9bc1-cb3a020c6df8 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:07.653981 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e53aa-c67c-4e32-9bc1-cb3a020c6df8 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:07.654185 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e53aa-c67c-4e32-9bc1-cb3a020c6df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:07.656442 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:07.687612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-ec1e53aa-c67c-4e32-9bc1-cb3a020c6df8 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:07.688851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-ec1e53aa-c67c-4e32-9bc1-cb3a020c6df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:07.769498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0359610-9482-490e-901b-af80dfc4254d None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:07.812911 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbd9437-26b4-42db-9617-ec092af1f19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:07.814650 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd9437-26b4-42db-9617-ec092af1f19c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:07.815171 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd9437-26b4-42db-9617-ec092af1f19c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:07.815662 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd9437-26b4-42db-9617-ec092af1f19c None admin] PATH_INFO: `/v3/projects/c631b5a3911e4647a05ad6138d2bc618` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:07.817454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbd9437-26b4-42db-9617-ec092af1f19c None admin] RBAC: Authorizing `identity:get_project(project_id=c631b5a3911e4647a05ad6138d2bc618)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:07.819349 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbd9437-26b4-42db-9617-ec092af1f19c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:07.822679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 211/415] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:19:07 2022] GET /identity/v3/projects/c631b5a3911e4647a05ad6138d2bc618 => generated 344 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:07.828776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbd9437-26b4-42db-9617-ec092af1f19c None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:07.877672 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 205/416] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:07.917846 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e53aa-c67c-4e32-9bc1-cb3a020c6df8 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:07.931799 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:07.958570 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbd9437-26b4-42db-9617-ec092af1f19c None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:07.972574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:08.009071 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f26dcd-fd35-47b7-9977-9aebd3fa9963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:08.011767 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f26dcd-fd35-47b7-9977-9aebd3fa9963 None admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:08.012347 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f26dcd-fd35-47b7-9977-9aebd3fa9963 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:08.012837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f26dcd-fd35-47b7-9977-9aebd3fa9963 None admin] PATH_INFO: `/v3/projects/c631b5a3911e4647a05ad6138d2bc618/groups/2b059d0198ff417d94197b72d0594da1/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:08.015057 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f26dcd-fd35-47b7-9977-9aebd3fa9963 None admin] RBAC: Authorizing `identity:create_grant(project_id=c631b5a3911e4647a05ad6138d2bc618, group_id=2b059d0198ff417d94197b72d0594da1, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:08.017285 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f26dcd-fd35-47b7-9977-9aebd3fa9963 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:08.038746 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 212/417] 10.222.0.16 () {58 vars in 1442 bytes} [Mon May 23 12:19:07 2022] PUT /identity/v3/projects/c631b5a3911e4647a05ad6138d2bc618/groups/2b059d0198ff417d94197b72d0594da1/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:19:08.059469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e53aa-c67c-4e32-9bc1-cb3a020c6df8 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:08.100722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438351c-0507-477a-96ac-60a276e2aa71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:08.101454 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438351c-0507-477a-96ac-60a276e2aa71 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:08.101756 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438351c-0507-477a-96ac-60a276e2aa71 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:08.102016 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438351c-0507-477a-96ac-60a276e2aa71 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:08.113688 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8438351c-0507-477a-96ac-60a276e2aa71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:08.122107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-8438351c-0507-477a-96ac-60a276e2aa71 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 12:19:08.122107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:08.122107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:08.122107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:08.122107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:08.122107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:08.122107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:08.122107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:08.122107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:08.122107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:08.122107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:08.122107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:08.122107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:08.122107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:08.122107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:08.122107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:08.123110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:08.123110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:08.123110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:08.123110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:08.123110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:08.123110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:08.123110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:08.123110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:08.123110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:08.123110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:08.123110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:08.123110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:08.123110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:08.123110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:08.123110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:08.123110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:08.123918 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:08.123918 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:08.123918 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:08.123918 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:08.123918 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:08.123918 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:08.123918 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:08.123918 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:08.123918 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:08.123918 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:08.123918 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 12:19:08.123918 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:08.123918 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 206/418] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 12:19:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:19:08.129515 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f26dcd-fd35-47b7-9977-9aebd3fa9963 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:08.276247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f26dcd-fd35-47b7-9977-9aebd3fa9963 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:08.315102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16f3fd3-d462-455a-b81f-d08751fb2e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:08.316220 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f3fd3-d462-455a-b81f-d08751fb2e47 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:08.316985 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f3fd3-d462-455a-b81f-d08751fb2e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:08.317467 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f3fd3-d462-455a-b81f-d08751fb2e47 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:08.318981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16f3fd3-d462-455a-b81f-d08751fb2e47 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:08.319594 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16f3fd3-d462-455a-b81f-d08751fb2e47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:08.321355 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16f3fd3-d462-455a-b81f-d08751fb2e47 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:08.331296 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 213/419] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:08.340303 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438351c-0507-477a-96ac-60a276e2aa71 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:08.471043 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:08.474996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438351c-0507-477a-96ac-60a276e2aa71 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:08.512778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e51ea6-e35d-45c0-8023-7c20728f8546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:08.513453 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e51ea6-e35d-45c0-8023-7c20728f8546 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:08.513741 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e51ea6-e35d-45c0-8023-7c20728f8546 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:08.514020 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e51ea6-e35d-45c0-8023-7c20728f8546 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:08.614807 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e51ea6-e35d-45c0-8023-7c20728f8546 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:08.616655 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e51ea6-e35d-45c0-8023-7c20728f8546 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:08.715399 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3ddb0-2563-4c3f-ba13-95f45ceb32fb None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:08.715736 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3ddb0-2563-4c3f-ba13-95f45ceb32fb None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:08.715970 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe3ddb0-2563-4c3f-ba13-95f45ceb32fb None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:08.717622 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 214/420] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:08.722721 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ca6f8-fd8c-4da7-a620-3d59381bd65a None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:08.722950 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ca6f8-fd8c-4da7-a620-3d59381bd65a None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:08.723237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ca6f8-fd8c-4da7-a620-3d59381bd65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:08.730647 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:08.741981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-897ca6f8-fd8c-4da7-a620-3d59381bd65a None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:08.743832 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-897ca6f8-fd8c-4da7-a620-3d59381bd65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:08.768478 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:08.777514 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 207/421] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:19:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:19:08.783943 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e51ea6-e35d-45c0-8023-7c20728f8546 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:08.875640 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 215/422] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 12:19:08.928086 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e51ea6-e35d-45c0-8023-7c20728f8546 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:08.954144 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ca6f8-fd8c-4da7-a620-3d59381bd65a None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:08.979780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c28144-f685-43b6-b01e-5334de21c3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:08.980596 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c28144-f685-43b6-b01e-5334de21c3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:08.980876 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c28144-f685-43b6-b01e-5334de21c3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:08.981151 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c28144-f685-43b6-b01e-5334de21c3e9 None admin] PATH_INFO: `/v3/projects/2bb553c7fe6e4284b56d30c136b7a041` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:08.984063 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c28144-f685-43b6-b01e-5334de21c3e9 None admin] RBAC: Authorizing `identity:get_project(project_id=2bb553c7fe6e4284b56d30c136b7a041)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:08.985593 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c28144-f685-43b6-b01e-5334de21c3e9 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:08.996783 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 208/423] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:19:08 2022] GET /identity/v3/projects/2bb553c7fe6e4284b56d30c136b7a041 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:08.997692 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c28144-f685-43b6-b01e-5334de21c3e9 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:09.126042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ca6f8-fd8c-4da7-a620-3d59381bd65a None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:09.164074 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857cd687-9dd7-4038-af53-bf4109302e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:09.165236 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cd687-9dd7-4038-af53-bf4109302e6c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:09.168370 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:09.168943 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cd687-9dd7-4038-af53-bf4109302e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:09.169233 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cd687-9dd7-4038-af53-bf4109302e6c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:09.186850 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:09.197712 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857cd687-9dd7-4038-af53-bf4109302e6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:09.213034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-857cd687-9dd7-4038-af53-bf4109302e6c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 12:19:09.213034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:09.213034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:09.213034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:09.213034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:09.213034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:09.213034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:09.213034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:09.213034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:09.213034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:09.213034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:09.213034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:09.213034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:09.213034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:09.213034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:09.213034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:09.213978 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:09.213978 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:09.213978 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:09.213978 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:09.213978 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:09.213978 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:09.213978 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:09.213978 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:09.213978 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:09.213978 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:09.213978 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:09.213978 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:09.213978 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:09.213978 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:09.213978 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:09.213978 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:09.214785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:09.214785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:09.214785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:09.214785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:09.214785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:09.214785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:09.214785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:09.214785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:09.214785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:09.214785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:09.214785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 12:19:09.214785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:09.214785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 216/424] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:19:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:09.220624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857cd687-9dd7-4038-af53-bf4109302e6c None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:09.290273 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c28144-f685-43b6-b01e-5334de21c3e9 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:09.336214 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857cd687-9dd7-4038-af53-bf4109302e6c None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:09.373891 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80331bf9-a1e9-40a8-9301-7e93ed7f5dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:09.374692 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80331bf9-a1e9-40a8-9301-7e93ed7f5dea None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:09.374981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80331bf9-a1e9-40a8-9301-7e93ed7f5dea None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:09.375257 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80331bf9-a1e9-40a8-9301-7e93ed7f5dea None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:09.376003 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80331bf9-a1e9-40a8-9301-7e93ed7f5dea None admin] RBAC: Adding query filter params (name=member) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:09.376367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80331bf9-a1e9-40a8-9301-7e93ed7f5dea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:09.378089 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80331bf9-a1e9-40a8-9301-7e93ed7f5dea None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:09.387629 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 217/425] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 12:19:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:09.393146 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80331bf9-a1e9-40a8-9301-7e93ed7f5dea None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:09.450287 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b99db2-699e-42b3-ba6b-c1bb8cded381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:09.451160 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b99db2-699e-42b3-ba6b-c1bb8cded381 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:09.451447 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b99db2-699e-42b3-ba6b-c1bb8cded381 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:09.451725 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b99db2-699e-42b3-ba6b-c1bb8cded381 None admin] PATH_INFO: `/v3/groups/b14634a2a38a44db8071dd00d49ded4a` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:09.452511 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:09.456861 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b99db2-699e-42b3-ba6b-c1bb8cded381 None admin] RBAC: Authorizing `identity:get_group(group_id=b14634a2a38a44db8071dd00d49ded4a)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:09.458341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b99db2-699e-42b3-ba6b-c1bb8cded381 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:09.461149 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 209/426] 10.222.0.16 () {56 vars in 1180 bytes} [Mon May 23 12:19:08 2022] GET /identity/v3/groups/b14634a2a38a44db8071dd00d49ded4a => generated 226 bytes in 464 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:09.469151 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b99db2-699e-42b3-ba6b-c1bb8cded381 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:09.514828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80331bf9-a1e9-40a8-9301-7e93ed7f5dea None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:09.557612 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc71cf0-31a1-42c6-a139-61bd8d2b4cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:09.558191 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc71cf0-31a1-42c6-a139-61bd8d2b4cfd None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:09.558883 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc71cf0-31a1-42c6-a139-61bd8d2b4cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:09.559167 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc71cf0-31a1-42c6-a139-61bd8d2b4cfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:09.610119 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b99db2-699e-42b3-ba6b-c1bb8cded381 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:09.645105 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc71cf0-31a1-42c6-a139-61bd8d2b4cfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:09.647003 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc71cf0-31a1-42c6-a139-61bd8d2b4cfd None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:09.654436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60edae4b-bc3e-428b-8268-8e08b0704c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:09.655099 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60edae4b-bc3e-428b-8268-8e08b0704c50 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:09.655903 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60edae4b-bc3e-428b-8268-8e08b0704c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:09.656235 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60edae4b-bc3e-428b-8268-8e08b0704c50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:09.657082 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60edae4b-bc3e-428b-8268-8e08b0704c50 None admin] RBAC: Adding query filter params (group.id=b14634a2a38a44db8071dd00d49ded4a, role.id=2d4732dacad6444e86d7fee3514afefb, scope.project.id=2bb553c7fe6e4284b56d30c136b7a041) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:09.657401 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60edae4b-bc3e-428b-8268-8e08b0704c50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:09.659318 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60edae4b-bc3e-428b-8268-8e08b0704c50 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:09.668519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 210/427] 10.222.0.16 () {56 vars in 1376 bytes} [Mon May 23 12:19:09 2022] GET /identity/v3/role_assignments?group.id=b14634a2a38a44db8071dd00d49ded4a&scope.project.id=2bb553c7fe6e4284b56d30c136b7a041&role.id=2d4732dacad6444e86d7fee3514afefb => generated 611 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:09.669921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3ad7b-28de-477c-9a1b-a958bb572848 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:09.670213 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3ad7b-28de-477c-9a1b-a958bb572848 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:09.670493 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3ad7b-28de-477c-9a1b-a958bb572848 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:09.671855 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 211/428] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:09.672497 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:09.694270 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652f3e8-2f02-4f2b-8ae9-772b1a516ae2 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:09.694571 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652f3e8-2f02-4f2b-8ae9-772b1a516ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:09.695013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652f3e8-2f02-4f2b-8ae9-772b1a516ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:09.722796 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:09.727365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-e652f3e8-2f02-4f2b-8ae9-772b1a516ae2 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:09.728756 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-e652f3e8-2f02-4f2b-8ae9-772b1a516ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:09.776021 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 218/429] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:19:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 12:19:09.786710 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc71cf0-31a1-42c6-a139-61bd8d2b4cfd None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:09.819296 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 212/430] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:09.862257 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652f3e8-2f02-4f2b-8ae9-772b1a516ae2 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:09.973451 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc71cf0-31a1-42c6-a139-61bd8d2b4cfd None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:10.016819 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652f3e8-2f02-4f2b-8ae9-772b1a516ae2 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:10.029330 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e7973b-3a7b-4306-8950-8616cb0c5492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:10.030503 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7973b-3a7b-4306-8950-8616cb0c5492 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:10.030783 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7973b-3a7b-4306-8950-8616cb0c5492 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:10.031059 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7973b-3a7b-4306-8950-8616cb0c5492 None admin] PATH_INFO: `/v3/projects/2bb553c7fe6e4284b56d30c136b7a041` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:10.036926 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e7973b-3a7b-4306-8950-8616cb0c5492 None admin] RBAC: Authorizing `identity:get_project(project_id=2bb553c7fe6e4284b56d30c136b7a041)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:10.038502 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e7973b-3a7b-4306-8950-8616cb0c5492 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:10.040566 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 219/431] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:19:09 2022] GET /identity/v3/projects/2bb553c7fe6e4284b56d30c136b7a041 => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:10.045489 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e7973b-3a7b-4306-8950-8616cb0c5492 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:10.061329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee58ac5-fe74-4da3-ad19-49a6cb040973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:10.061329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee58ac5-fe74-4da3-ad19-49a6cb040973 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:10.061329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee58ac5-fe74-4da3-ad19-49a6cb040973 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:10.061329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee58ac5-fe74-4da3-ad19-49a6cb040973 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:10.072268 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee58ac5-fe74-4da3-ad19-49a6cb040973 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:10.078664 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:10.080663 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:10.083284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-1ee58ac5-fe74-4da3-ad19-49a6cb040973 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 12:19:10.083284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:10.083284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:10.083284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:10.083284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:10.083284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:10.083284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:10.083284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:10.083284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:10.083284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:10.083284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:10.083284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:10.083284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:10.083284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:10.083284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:10.083284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:10.084855 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:10.084855 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:10.084855 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:10.084855 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:10.084855 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:10.084855 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:10.084855 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:10.084855 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:10.084855 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:10.084855 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:10.084855 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:10.084855 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:10.084855 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:10.084855 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:10.084855 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:10.084855 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:10.085976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:10.085976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:10.085976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:10.085976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:10.085976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:10.085976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:10.085976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:10.085976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:10.085976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:10.085976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:10.085976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 12:19:10.085976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:10.085976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 213/432] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:19:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:19:10.098163 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee58ac5-fe74-4da3-ad19-49a6cb040973 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:10.178883 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e7973b-3a7b-4306-8950-8616cb0c5492 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:10.227801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f70e7-c5b3-4d32-bff0-c96ff7193241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:10.228578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee58ac5-fe74-4da3-ad19-49a6cb040973 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:10.228689 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f70e7-c5b3-4d32-bff0-c96ff7193241 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:10.228974 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f70e7-c5b3-4d32-bff0-c96ff7193241 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:10.229253 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8f70e7-c5b3-4d32-bff0-c96ff7193241 None admin] PATH_INFO: `/v3/groups/b14634a2a38a44db8071dd00d49ded4a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:10.230440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8f70e7-c5b3-4d32-bff0-c96ff7193241 None admin] RBAC: Authorizing `identity:get_group(group_id=b14634a2a38a44db8071dd00d49ded4a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:10.233105 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8f70e7-c5b3-4d32-bff0-c96ff7193241 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:10.235047 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 220/433] 10.222.0.16 () {56 vars in 1180 bytes} [Mon May 23 12:19:10 2022] GET /identity/v3/groups/b14634a2a38a44db8071dd00d49ded4a => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:19:10.241524 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f70e7-c5b3-4d32-bff0-c96ff7193241 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:10.269515 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e609c77-c5a7-42a2-82da-5030121e9e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:10.270537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e609c77-c5a7-42a2-82da-5030121e9e5b None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:10.271020 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e609c77-c5a7-42a2-82da-5030121e9e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:10.271474 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e609c77-c5a7-42a2-82da-5030121e9e5b None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:10.273101 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e609c77-c5a7-42a2-82da-5030121e9e5b None admin] RBAC: Adding query filter params (name=member) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:10.273479 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e609c77-c5a7-42a2-82da-5030121e9e5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:10.274955 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e609c77-c5a7-42a2-82da-5030121e9e5b None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:10.288675 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 214/434] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 12:19:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:10.289751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e609c77-c5a7-42a2-82da-5030121e9e5b None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:10.348116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:10.370541 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:10.382358 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8f70e7-c5b3-4d32-bff0-c96ff7193241 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:10.403006 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e609c77-c5a7-42a2-82da-5030121e9e5b None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:10.421360 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe180f19-7830-420a-9976-d02cb3d82e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:10.422022 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe180f19-7830-420a-9976-d02cb3d82e02 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:10.422312 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe180f19-7830-420a-9976-d02cb3d82e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:10.422720 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe180f19-7830-420a-9976-d02cb3d82e02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:10.423551 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe180f19-7830-420a-9976-d02cb3d82e02 None admin] RBAC: Adding query filter params (group.id=b14634a2a38a44db8071dd00d49ded4a, role.id=504a45d998f0464893c90fa5429275a8, scope.project.id=2bb553c7fe6e4284b56d30c136b7a041) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:10.423873 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe180f19-7830-420a-9976-d02cb3d82e02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:10.425477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe180f19-7830-420a-9976-d02cb3d82e02 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:10.432805 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 221/435] 10.222.0.16 () {56 vars in 1376 bytes} [Mon May 23 12:19:10 2022] GET /identity/v3/role_assignments?group.id=b14634a2a38a44db8071dd00d49ded4a&scope.project.id=2bb553c7fe6e4284b56d30c136b7a041&role.id=504a45d998f0464893c90fa5429275a8 => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:10.434155 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbc966-3181-4b21-b1a0-87bb6e55f3f0 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:10.434452 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbc966-3181-4b21-b1a0-87bb6e55f3f0 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:10.434735 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbc966-3181-4b21-b1a0-87bb6e55f3f0 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:10.436151 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 222/436] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:10.442150 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49948f-8f91-42c2-ab0e-14df0de8a201 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:10.442441 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49948f-8f91-42c2-ab0e-14df0de8a201 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:10.442725 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49948f-8f91-42c2-ab0e-14df0de8a201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:10.446180 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398652bb-cf62-4950-a3d8-87574b1aa9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:10.447157 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398652bb-cf62-4950-a3d8-87574b1aa9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:10.447620 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398652bb-cf62-4950-a3d8-87574b1aa9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:10.448070 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398652bb-cf62-4950-a3d8-87574b1aa9d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:10.462900 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-de49948f-8f91-42c2-ab0e-14df0de8a201 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:10.464202 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-de49948f-8f91-42c2-ab0e-14df0de8a201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:10.529064 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398652bb-cf62-4950-a3d8-87574b1aa9d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:10.530954 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398652bb-cf62-4950-a3d8-87574b1aa9d0 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:10.568902 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 223/437] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 12:19:10.595971 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de49948f-8f91-42c2-ab0e-14df0de8a201 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:10.601689 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:10.666592 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 215/438] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:19:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:19:10.674398 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398652bb-cf62-4950-a3d8-87574b1aa9d0 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:10.713579 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:10.735657 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de49948f-8f91-42c2-ab0e-14df0de8a201 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:10.777604 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d60aa-db8d-4b47-a172-cbc8a7901a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:10.778406 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d60aa-db8d-4b47-a172-cbc8a7901a7c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:10.778996 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d60aa-db8d-4b47-a172-cbc8a7901a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:10.778996 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d60aa-db8d-4b47-a172-cbc8a7901a7c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:10.790141 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492d60aa-db8d-4b47-a172-cbc8a7901a7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:10.799250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-492d60aa-db8d-4b47-a172-cbc8a7901a7c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 12:19:10.799250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:10.799250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:10.799250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:10.799250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:10.799250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:10.799250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:10.799250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:10.799250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:10.799250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:10.799250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:10.799250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:10.799250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:10.799250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:10.799250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:10.799250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:10.800273 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:10.800273 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:10.800273 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:10.800273 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:10.800273 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:10.800273 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:10.800273 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:10.800273 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:10.800273 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:10.800273 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:10.800273 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:10.800273 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:10.800273 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:10.800273 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:10.800273 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:10.800273 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:10.801075 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:10.801075 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:10.801075 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:10.801075 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:10.801075 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:10.801075 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:10.801075 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:10.801075 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:10.801075 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:10.801075 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:10.801075 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 12:19:10.801075 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:10.801075 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 224/439] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 12:19:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:19:10.807625 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d60aa-db8d-4b47-a172-cbc8a7901a7c None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:10.810857 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398652bb-cf62-4950-a3d8-87574b1aa9d0 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:10.851599 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843b60b-422a-449f-8895-6d7bbd8bd6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:10.853026 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843b60b-422a-449f-8895-6d7bbd8bd6fc None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:10.853264 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843b60b-422a-449f-8895-6d7bbd8bd6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:10.853540 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843b60b-422a-449f-8895-6d7bbd8bd6fc None admin] PATH_INFO: `/v3/projects/49c3998144de4918b868b834ea2918dc` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:10.854762 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3843b60b-422a-449f-8895-6d7bbd8bd6fc None admin] RBAC: Authorizing `identity:get_project(project_id=49c3998144de4918b868b834ea2918dc)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:10.856260 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3843b60b-422a-449f-8895-6d7bbd8bd6fc None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:10.857957 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 216/440] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:19:10 2022] GET /identity/v3/projects/49c3998144de4918b868b834ea2918dc => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:10.859287 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83bf99-1b26-4f0b-9a87-59968ee7cb19 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:10.859581 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83bf99-1b26-4f0b-9a87-59968ee7cb19 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:10.859864 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83bf99-1b26-4f0b-9a87-59968ee7cb19 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:10.861119 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 217/441] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:10.863358 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83bf99-1b26-4f0b-9a87-59968ee7cb19 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:10.893726 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:10.936655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d60aa-db8d-4b47-a172-cbc8a7901a7c None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:10.978795 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c263b-f50d-41ee-90e5-78e1aa331d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:10.979473 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c263b-f50d-41ee-90e5-78e1aa331d39 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:10.979759 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c263b-f50d-41ee-90e5-78e1aa331d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:10.980038 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c263b-f50d-41ee-90e5-78e1aa331d39 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:10.980829 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c263b-f50d-41ee-90e5-78e1aa331d39 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:10.981141 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c263b-f50d-41ee-90e5-78e1aa331d39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:10.982522 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c263b-f50d-41ee-90e5-78e1aa331d39 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:10.989354 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83bf99-1b26-4f0b-9a87-59968ee7cb19 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:10.990948 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 225/442] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:19:10.992275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd1e5b-5a09-4ddd-beba-e236b20f999f None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:10.992570 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd1e5b-5a09-4ddd-beba-e236b20f999f None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:10.992851 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd1e5b-5a09-4ddd-beba-e236b20f999f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:11.004605 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:11.010288 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-15bd1e5b-5a09-4ddd-beba-e236b20f999f None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:11.011482 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-15bd1e5b-5a09-4ddd-beba-e236b20f999f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:11.035120 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9acbb-5621-4230-b957-e7935e835560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:11.036719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9acbb-5621-4230-b957-e7935e835560 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:11.036719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9acbb-5621-4230-b957-e7935e835560 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:11.036719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d9acbb-5621-4230-b957-e7935e835560 None admin] PATH_INFO: `/v3/groups/b14634a2a38a44db8071dd00d49ded4a` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:11.038729 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9acbb-5621-4230-b957-e7935e835560 None admin] RBAC: Authorizing `identity:get_group(group_id=b14634a2a38a44db8071dd00d49ded4a)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:11.040242 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d9acbb-5621-4230-b957-e7935e835560 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:11.042688 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 218/443] 10.222.0.16 () {56 vars in 1180 bytes} [Mon May 23 12:19:10 2022] GET /identity/v3/groups/b14634a2a38a44db8071dd00d49ded4a => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:19:11.042688 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9acbb-5621-4230-b957-e7935e835560 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:11.096839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 226/444] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:11.097692 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd1e5b-5a09-4ddd-beba-e236b20f999f None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:11.141304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:11.154512 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d9acbb-5621-4230-b957-e7935e835560 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:11.187517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f50c9-8076-4c0d-a84a-184b161a028c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:11.188154 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f50c9-8076-4c0d-a84a-184b161a028c None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:11.188460 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f50c9-8076-4c0d-a84a-184b161a028c None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:11.188735 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f50c9-8076-4c0d-a84a-184b161a028c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:11.196903 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd1e5b-5a09-4ddd-beba-e236b20f999f None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:11.230378 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84146d-7281-4459-849f-ccaf2196382e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:11.231033 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84146d-7281-4459-849f-ccaf2196382e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:11.231318 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84146d-7281-4459-849f-ccaf2196382e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:11.231605 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84146d-7281-4459-849f-ccaf2196382e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:11.232462 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d84146d-7281-4459-849f-ccaf2196382e None admin] RBAC: Adding query filter params (group.id=b14634a2a38a44db8071dd00d49ded4a, role.id=504a45d998f0464893c90fa5429275a8, scope.project.id=49c3998144de4918b868b834ea2918dc) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:11.232785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d84146d-7281-4459-849f-ccaf2196382e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:11.234205 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d84146d-7281-4459-849f-ccaf2196382e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:11.236998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:11.244340 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 227/445] 10.222.0.16 () {56 vars in 1376 bytes} [Mon May 23 12:19:11 2022] GET /identity/v3/role_assignments?group.id=b14634a2a38a44db8071dd00d49ded4a&scope.project.id=49c3998144de4918b868b834ea2918dc&role.id=504a45d998f0464893c90fa5429275a8 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:11.245239 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84146d-7281-4459-849f-ccaf2196382e None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:11.258223 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f50c9-8076-4c0d-a84a-184b161a028c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:11.259778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f50c9-8076-4c0d-a84a-184b161a028c None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:11.358521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:11.362196 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84146d-7281-4459-849f-ccaf2196382e None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:11.380511 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 219/446] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:19:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:19:11.382088 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f50c9-8076-4c0d-a84a-184b161a028c None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:11.405313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc59533-38db-4247-9ccc-2323feb37d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:11.406416 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc59533-38db-4247-9ccc-2323feb37d56 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:11.406894 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc59533-38db-4247-9ccc-2323feb37d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:11.407351 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc59533-38db-4247-9ccc-2323feb37d56 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:11.419486 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc59533-38db-4247-9ccc-2323feb37d56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:11.427125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-ffc59533-38db-4247-9ccc-2323feb37d56 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:19:11.427125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:11.427125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:11.427125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:11.427125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:11.427125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:11.427125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:11.427125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:11.427125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:11.427125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:11.427125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:11.427125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:11.427125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:11.427125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:11.427125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:11.427125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:11.428094 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:11.428094 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:11.428094 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:11.428094 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:11.428094 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:11.428094 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:11.428094 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:11.428094 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:11.428094 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:11.428094 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:11.428094 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:11.428094 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:11.428094 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:11.428094 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:11.428094 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:11.428094 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:11.429287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:11.429287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:11.429287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:11.429287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:11.429287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:11.429287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:11.429287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:11.429287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:11.429287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:11.429287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:11.429287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:19:11.429287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:11.429287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 228/447] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:19:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:11.434698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc59533-38db-4247-9ccc-2323feb37d56 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:11.505363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f50c9-8076-4c0d-a84a-184b161a028c None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:11.512503 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:11.543764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8969c30a-9adb-4572-8014-796f91db222a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:11.544946 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969c30a-9adb-4572-8014-796f91db222a None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:11.545374 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969c30a-9adb-4572-8014-796f91db222a None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:11.546257 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969c30a-9adb-4572-8014-796f91db222a None admin] PATH_INFO: `/v3/projects/49c3998144de4918b868b834ea2918dc` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:11.547765 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8969c30a-9adb-4572-8014-796f91db222a None admin] RBAC: Authorizing `identity:get_project(project_id=49c3998144de4918b868b834ea2918dc)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:11.549276 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8969c30a-9adb-4572-8014-796f91db222a None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:11.551144 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 220/448] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:19:11 2022] GET /identity/v3/projects/49c3998144de4918b868b834ea2918dc => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:19:11.557134 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8969c30a-9adb-4572-8014-796f91db222a None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:11.559920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc59533-38db-4247-9ccc-2323feb37d56 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:11.602628 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fd94a-0a63-40e6-92a0-aa023f1fc0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:11.603309 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fd94a-0a63-40e6-92a0-aa023f1fc0c7 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:11.603599 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fd94a-0a63-40e6-92a0-aa023f1fc0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:11.603882 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fd94a-0a63-40e6-92a0-aa023f1fc0c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:11.604722 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73fd94a-0a63-40e6-92a0-aa023f1fc0c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:11.605038 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73fd94a-0a63-40e6-92a0-aa023f1fc0c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:11.607138 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73fd94a-0a63-40e6-92a0-aa023f1fc0c7 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:11.617842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 229/449] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:19:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:19:11.623280 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:11.623588 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fd94a-0a63-40e6-92a0-aa023f1fc0c7 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:11.670698 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8969c30a-9adb-4572-8014-796f91db222a None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:11.704982 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c80c33-ed70-4b94-9c6b-5afba021559d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:11.705830 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c80c33-ed70-4b94-9c6b-5afba021559d None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:11.706114 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c80c33-ed70-4b94-9c6b-5afba021559d None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:11.706392 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c80c33-ed70-4b94-9c6b-5afba021559d None admin] PATH_INFO: `/v3/groups/b14634a2a38a44db8071dd00d49ded4a` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:11.707560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c80c33-ed70-4b94-9c6b-5afba021559d None admin] RBAC: Authorizing `identity:get_group(group_id=b14634a2a38a44db8071dd00d49ded4a)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:11.709142 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c80c33-ed70-4b94-9c6b-5afba021559d None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:11.710749 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 221/450] 10.222.0.16 () {56 vars in 1180 bytes} [Mon May 23 12:19:11 2022] GET /identity/v3/groups/b14634a2a38a44db8071dd00d49ded4a => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:11.716423 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c80c33-ed70-4b94-9c6b-5afba021559d None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:11.725603 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fd94a-0a63-40e6-92a0-aa023f1fc0c7 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:11.747782 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:11.760101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db255642-79ef-4b0f-9ea8-1c59c480c1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:11.760847 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db255642-79ef-4b0f-9ea8-1c59c480c1aa None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:11.761132 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db255642-79ef-4b0f-9ea8-1c59c480c1aa None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:11.761409 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db255642-79ef-4b0f-9ea8-1c59c480c1aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:11.814831 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c80c33-ed70-4b94-9c6b-5afba021559d None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:11.828570 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db255642-79ef-4b0f-9ea8-1c59c480c1aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:11.830037 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db255642-79ef-4b0f-9ea8-1c59c480c1aa None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:11.837967 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:11.848776 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd9d14-6af2-4f59-b23d-31e484ccc314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:11.849521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd9d14-6af2-4f59-b23d-31e484ccc314 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:11.850402 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd9d14-6af2-4f59-b23d-31e484ccc314 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:11.850813 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd9d14-6af2-4f59-b23d-31e484ccc314 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:11.851721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd9d14-6af2-4f59-b23d-31e484ccc314 None admin] RBAC: Adding query filter params (group.id=b14634a2a38a44db8071dd00d49ded4a, role.id=2d4732dacad6444e86d7fee3514afefb, scope.project.id=49c3998144de4918b868b834ea2918dc) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:11.852112 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd9d14-6af2-4f59-b23d-31e484ccc314 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:11.853621 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd9d14-6af2-4f59-b23d-31e484ccc314 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:11.859446 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 222/451] 10.222.0.16 () {56 vars in 1376 bytes} [Mon May 23 12:19:11 2022] GET /identity/v3/role_assignments?group.id=b14634a2a38a44db8071dd00d49ded4a&scope.project.id=49c3998144de4918b868b834ea2918dc&role.id=2d4732dacad6444e86d7fee3514afefb => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:19:11.933383 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 230/452] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:19:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 12:19:11.938928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd9d14-6af2-4f59-b23d-31e484ccc314 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:12.045443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd9d14-6af2-4f59-b23d-31e484ccc314 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:12.078707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074574b8-e17f-4a17-a29f-52f0acc5852b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:12.079564 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074574b8-e17f-4a17-a29f-52f0acc5852b None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:12.079850 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074574b8-e17f-4a17-a29f-52f0acc5852b None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:12.080137 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074574b8-e17f-4a17-a29f-52f0acc5852b None admin] PATH_INFO: `/v3/projects/c631b5a3911e4647a05ad6138d2bc618` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:12.081350 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074574b8-e17f-4a17-a29f-52f0acc5852b None admin] RBAC: Authorizing `identity:get_project(project_id=c631b5a3911e4647a05ad6138d2bc618)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:12.082786 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074574b8-e17f-4a17-a29f-52f0acc5852b None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:12.084530 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 223/453] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:19:11 2022] GET /identity/v3/projects/c631b5a3911e4647a05ad6138d2bc618 => generated 344 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:12.090048 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db255642-79ef-4b0f-9ea8-1c59c480c1aa None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:12.114557 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:12.195327 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db255642-79ef-4b0f-9ea8-1c59c480c1aa None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:12.229932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ae338-5c22-454c-b5f5-ee691c1e18a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:12.230782 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ae338-5c22-454c-b5f5-ee691c1e18a8 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:12.231065 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ae338-5c22-454c-b5f5-ee691c1e18a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:12.231347 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ae338-5c22-454c-b5f5-ee691c1e18a8 None admin] PATH_INFO: `/v3/groups/2b059d0198ff417d94197b72d0594da1` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:12.232538 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ae338-5c22-454c-b5f5-ee691c1e18a8 None admin] RBAC: Authorizing `identity:get_group(group_id=2b059d0198ff417d94197b72d0594da1)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:12.233973 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ae338-5c22-454c-b5f5-ee691c1e18a8 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:12.235591 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 231/454] 10.222.0.16 () {56 vars in 1180 bytes} [Mon May 23 12:19:12 2022] GET /identity/v3/groups/2b059d0198ff417d94197b72d0594da1 => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:12.241605 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074574b8-e17f-4a17-a29f-52f0acc5852b None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:12.296301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:12.338401 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074574b8-e17f-4a17-a29f-52f0acc5852b None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:12.370573 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd23b0-3477-4135-8d19-83f09c90c906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:12.371211 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd23b0-3477-4135-8d19-83f09c90c906 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:12.371497 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd23b0-3477-4135-8d19-83f09c90c906 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:12.371776 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd23b0-3477-4135-8d19-83f09c90c906 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:12.372628 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd23b0-3477-4135-8d19-83f09c90c906 None admin] RBAC: Adding query filter params (group.id=2b059d0198ff417d94197b72d0594da1, role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8, scope.project.id=c631b5a3911e4647a05ad6138d2bc618) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:12.372949 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd23b0-3477-4135-8d19-83f09c90c906 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:12.374389 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd23b0-3477-4135-8d19-83f09c90c906 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:12.379994 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 224/455] 10.222.0.16 () {56 vars in 1376 bytes} [Mon May 23 12:19:12 2022] GET /identity/v3/role_assignments?group.id=2b059d0198ff417d94197b72d0594da1&scope.project.id=c631b5a3911e4647a05ad6138d2bc618&role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:15.263357 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ac288-d495-445a-889e-26cb975c7d3d None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:15.264506 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ac288-d495-445a-889e-26cb975c7d3d None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:15.265153 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ac288-d495-445a-889e-26cb975c7d3d None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:15.267377 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 232/456] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:15.273064 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23774d-426c-45c0-bfdb-8c1d06d02905 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:15.274054 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23774d-426c-45c0-bfdb-8c1d06d02905 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:15.274548 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23774d-426c-45c0-bfdb-8c1d06d02905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:15.298114 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-1b23774d-426c-45c0-bfdb-8c1d06d02905 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:15.299724 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-1b23774d-426c-45c0-bfdb-8c1d06d02905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:15.409314 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 225/457] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:15.459399 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ac288-d495-445a-889e-26cb975c7d3d None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:15.595398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:15.609125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ac288-d495-445a-889e-26cb975c7d3d None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:15.654875 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c1856b-5871-4d68-a0dc-aff97fc601d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:15.656109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1856b-5871-4d68-a0dc-aff97fc601d5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:15.656636 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1856b-5871-4d68-a0dc-aff97fc601d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:15.657107 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1856b-5871-4d68-a0dc-aff97fc601d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:15.670819 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c1856b-5871-4d68-a0dc-aff97fc601d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:15.672610 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c1856b-5871-4d68-a0dc-aff97fc601d5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:15.675617 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3aa7d6-fa69-4bd3-9488-55ffbe5d54bf None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:15.675912 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3aa7d6-fa69-4bd3-9488-55ffbe5d54bf None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:15.676211 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3aa7d6-fa69-4bd3-9488-55ffbe5d54bf None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:15.678593 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 226/458] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:15.684078 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24d912-9aec-4539-ae32-952d2473e21c None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:15.684656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24d912-9aec-4539-ae32-952d2473e21c None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:15.685140 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24d912-9aec-4539-ae32-952d2473e21c None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:15.686906 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 227/459] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:15.688251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ea629-26e1-4f3c-910c-aaed388ec431 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:15.688750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ea629-26e1-4f3c-910c-aaed388ec431 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:15.689242 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ea629-26e1-4f3c-910c-aaed388ec431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:15.698877 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-74c1856b-5871-4d68-a0dc-aff97fc601d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:15.698877 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:15.698877 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:15.698877 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:15.698877 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:15.698877 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:15.698877 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:15.698877 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:15.698877 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:15.698877 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:15.698877 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:15.698877 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:15.698877 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:15.698877 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:15.698877 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:15.699785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:15.699785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:15.699785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:15.699785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:15.699785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:15.699785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:15.699785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:15.699785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:15.699785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:15.699785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:15.699785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:15.699785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:15.699785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:15.699785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:15.699785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:15.699785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:15.699785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:15.700679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:15.700679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:15.700679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:15.700679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:15.700679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 233/460] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:15.702305 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b5299-4b56-4b7b-a2b6-27ea0bbd383e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:15.702305 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b5299-4b56-4b7b-a2b6-27ea0bbd383e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:15.702305 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173b5299-4b56-4b7b-a2b6-27ea0bbd383e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:15.709431 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-845ea629-26e1-4f3c-910c-aaed388ec431 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:15.710888 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-845ea629-26e1-4f3c-910c-aaed388ec431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:15.721943 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-173b5299-4b56-4b7b-a2b6-27ea0bbd383e None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:15.723245 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-173b5299-4b56-4b7b-a2b6-27ea0bbd383e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:15.768933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:15.799302 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 228/462] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:15.800458 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ea629-26e1-4f3c-910c-aaed388ec431 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:15.808318 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 234/462] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:15.825879 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b5299-4b56-4b7b-a2b6-27ea0bbd383e None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:15.907497 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:15.937575 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ea629-26e1-4f3c-910c-aaed388ec431 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:15.938408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173b5299-4b56-4b7b-a2b6-27ea0bbd383e None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:15.977020 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1acd27-b307-4cb3-be5a-19fc2877bb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:15.977818 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1acd27-b307-4cb3-be5a-19fc2877bb8a None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:15.978094 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1acd27-b307-4cb3-be5a-19fc2877bb8a None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:15.978366 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1acd27-b307-4cb3-be5a-19fc2877bb8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:15.978656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d56705-3f5b-4467-83f8-5bc7f986ad04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:15.979315 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d56705-3f5b-4467-83f8-5bc7f986ad04 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:15.979600 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d56705-3f5b-4467-83f8-5bc7f986ad04 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:15.979878 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d56705-3f5b-4467-83f8-5bc7f986ad04 None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:15.980638 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d56705-3f5b-4467-83f8-5bc7f986ad04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:15.980945 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d56705-3f5b-4467-83f8-5bc7f986ad04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:15.982339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d56705-3f5b-4467-83f8-5bc7f986ad04 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:15.988955 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1acd27-b307-4cb3-be5a-19fc2877bb8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:15.991279 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1acd27-b307-4cb3-be5a-19fc2877bb8a None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:15.999167 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 229/463] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:15.999167 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d56705-3f5b-4467-83f8-5bc7f986ad04 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:16.011845 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-ee1acd27-b307-4cb3-be5a-19fc2877bb8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:16.011845 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:16.011845 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:16.011845 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:16.011845 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:16.011845 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:16.011845 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:16.011845 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:16.011845 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:16.011845 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:16.011845 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:16.011845 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:16.011845 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:16.011845 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:16.011845 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:16.012770 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:16.012770 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:16.012770 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:16.012770 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:16.012770 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:16.012770 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:16.012770 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:16.012770 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:16.012770 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:16.012770 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:16.012770 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:16.012770 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:16.012770 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:16.012770 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:16.012770 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:16.012770 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:16.012770 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:16.013617 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:16.013617 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:16.013617 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:16.013617 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:16.017691 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 235/464] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:19:16.019462 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b070f-a5d6-48ff-aab2-4f53168dbfc0 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:16.019751 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b070f-a5d6-48ff-aab2-4f53168dbfc0 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:16.020028 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b070f-a5d6-48ff-aab2-4f53168dbfc0 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:16.021397 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 236/465] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:16.023315 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b070f-a5d6-48ff-aab2-4f53168dbfc0 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:16.050313 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:16.113819 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d56705-3f5b-4467-83f8-5bc7f986ad04 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:16.141221 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b070f-a5d6-48ff-aab2-4f53168dbfc0 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:16.147368 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735eafd-7343-4f4b-bebd-17665dda2d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:16.148162 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735eafd-7343-4f4b-bebd-17665dda2d1d None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:16.148476 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735eafd-7343-4f4b-bebd-17665dda2d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:16.148755 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735eafd-7343-4f4b-bebd-17665dda2d1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:16.159398 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735eafd-7343-4f4b-bebd-17665dda2d1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:16.160813 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735eafd-7343-4f4b-bebd-17665dda2d1d None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:16.166819 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:16.178665 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-8735eafd-7343-4f4b-bebd-17665dda2d1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:16.178665 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:16.178665 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:16.178665 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:16.178665 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:16.178665 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:16.178665 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:16.178665 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:16.178665 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:16.178665 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:16.178665 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:16.178665 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:16.178665 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:16.178665 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:16.178665 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:16.179562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:16.179562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:16.179562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:16.179562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:16.179562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:16.179562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:16.179562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:16.179562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:16.179562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:16.179562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:16.179562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:16.179562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:16.179562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:16.179562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:16.179562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:16.179562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:16.179562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:16.181182 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:16.181182 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:16.181182 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:16.181182 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:16.181182 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 230/466] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:16.181479 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1975f59-6568-4ff0-958e-ac2eb033abc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:16.181565 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1975f59-6568-4ff0-958e-ac2eb033abc6 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:16.181821 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1975f59-6568-4ff0-958e-ac2eb033abc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:16.182266 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1975f59-6568-4ff0-958e-ac2eb033abc6 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:16.182574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735eafd-7343-4f4b-bebd-17665dda2d1d None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:16.183252 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1975f59-6568-4ff0-958e-ac2eb033abc6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:16.185035 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1975f59-6568-4ff0-958e-ac2eb033abc6 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:16.206772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 237/467] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 12:19:16 2022] POST /identity/v3/users => generated 248 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:19:16.208081 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c85f88-5b01-402b-8b9c-e90ae0b3cb5b None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:16.208451 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c85f88-5b01-402b-8b9c-e90ae0b3cb5b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:16.208740 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c85f88-5b01-402b-8b9c-e90ae0b3cb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:16.233163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-30c85f88-5b01-402b-8b9c-e90ae0b3cb5b None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:16.234401 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-30c85f88-5b01-402b-8b9c-e90ae0b3cb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:16.302440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:16.315089 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735eafd-7343-4f4b-bebd-17665dda2d1d None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:16.328744 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 238/468] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:16.328744 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c85f88-5b01-402b-8b9c-e90ae0b3cb5b None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:16.370349 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6986348d-f360-465b-aeaf-330786858b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:16.371021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986348d-f360-465b-aeaf-330786858b92 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:16.371563 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986348d-f360-465b-aeaf-330786858b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:16.371836 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6986348d-f360-465b-aeaf-330786858b92 None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:16.372669 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6986348d-f360-465b-aeaf-330786858b92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:16.373036 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6986348d-f360-465b-aeaf-330786858b92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:16.374443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6986348d-f360-465b-aeaf-330786858b92 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:16.390129 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 231/469] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:16.391482 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6986348d-f360-465b-aeaf-330786858b92 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:16.455867 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c85f88-5b01-402b-8b9c-e90ae0b3cb5b None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:16.472384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:16.509957 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd4185-d3c3-4a37-a399-1ad52e6dfdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:16.510626 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd4185-d3c3-4a37-a399-1ad52e6dfdcf None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:16.510911 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd4185-d3c3-4a37-a399-1ad52e6dfdcf None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:16.511185 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd4185-d3c3-4a37-a399-1ad52e6dfdcf None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:16.511913 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afd4185-d3c3-4a37-a399-1ad52e6dfdcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:16.512237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afd4185-d3c3-4a37-a399-1ad52e6dfdcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:16.513619 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afd4185-d3c3-4a37-a399-1ad52e6dfdcf None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:16.518833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6986348d-f360-465b-aeaf-330786858b92 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:16.526966 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 239/470] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:16.527693 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd4185-d3c3-4a37-a399-1ad52e6dfdcf None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:16.556391 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dba8c5-ace8-40b9-87a2-79daa00a68c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:16.557191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dba8c5-ace8-40b9-87a2-79daa00a68c9 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:16.557479 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dba8c5-ace8-40b9-87a2-79daa00a68c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:16.557789 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dba8c5-ace8-40b9-87a2-79daa00a68c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:16.569814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dba8c5-ace8-40b9-87a2-79daa00a68c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:16.570717 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dba8c5-ace8-40b9-87a2-79daa00a68c9 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:16.589272 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-45dba8c5-ace8-40b9-87a2-79daa00a68c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:16.589272 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:16.589272 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:16.589272 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:16.589272 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:16.589272 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:16.589272 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:16.589272 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:16.589272 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:16.589272 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:16.589272 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:16.589272 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:16.589272 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:16.589272 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:16.589272 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:16.590173 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:16.590173 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:16.590173 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:16.590173 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:16.590173 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:16.590173 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:16.590173 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:16.590173 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:16.590173 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:16.590173 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:16.590173 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:16.590173 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:16.590173 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:16.590173 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:16.590173 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:16.590173 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:16.590173 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:16.591081 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:16.591081 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:16.591081 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:16.591081 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:16.591081 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 232/471] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:16.591932 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a60db3-2cbe-4e86-bce0-a43d6fe45c39 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:16.592032 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a60db3-2cbe-4e86-bce0-a43d6fe45c39 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:16.592300 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a60db3-2cbe-4e86-bce0-a43d6fe45c39 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:16.593682 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 233/472] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:16.594223 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a60db3-2cbe-4e86-bce0-a43d6fe45c39 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:16.598882 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:16.650176 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd4185-d3c3-4a37-a399-1ad52e6dfdcf None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:16.687081 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca50bc-349a-422f-ae0c-08d063e1bfd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:16.687764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca50bc-349a-422f-ae0c-08d063e1bfd3 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:16.688048 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca50bc-349a-422f-ae0c-08d063e1bfd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:16.688365 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca50bc-349a-422f-ae0c-08d063e1bfd3 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:16.689067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eca50bc-349a-422f-ae0c-08d063e1bfd3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:16.690509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eca50bc-349a-422f-ae0c-08d063e1bfd3 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:16.704939 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a60db3-2cbe-4e86-bce0-a43d6fe45c39 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:16.714228 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 240/473] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 12:19:16 2022] POST /identity/v3/users => generated 248 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:19:16.715232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca50bc-349a-422f-ae0c-08d063e1bfd3 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:16.760554 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06458352-9aa4-4044-801e-a27371cecb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:16.761553 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06458352-9aa4-4044-801e-a27371cecb74 None admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:16.762023 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06458352-9aa4-4044-801e-a27371cecb74 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:16.762449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06458352-9aa4-4044-801e-a27371cecb74 None admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:16.763100 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:16.764876 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06458352-9aa4-4044-801e-a27371cecb74 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:16.766906 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06458352-9aa4-4044-801e-a27371cecb74 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:16.800557 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 234/474] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 12:19:16 2022] POST /identity/v3/users => generated 246 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:19:16.802487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521669be-58ee-4a3b-ba76-5ceb2fda7efd None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:16.803227 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521669be-58ee-4a3b-ba76-5ceb2fda7efd None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:16.803690 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521669be-58ee-4a3b-ba76-5ceb2fda7efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:16.824403 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-521669be-58ee-4a3b-ba76-5ceb2fda7efd None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:16.825990 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-521669be-58ee-4a3b-ba76-5ceb2fda7efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:16.853354 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca50bc-349a-422f-ae0c-08d063e1bfd3 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:16.891682 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c314103-3a39-445b-8440-984299190fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:16.892497 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c314103-3a39-445b-8440-984299190fae None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:16.892875 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c314103-3a39-445b-8440-984299190fae None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:16.893223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c314103-3a39-445b-8440-984299190fae None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:16.894043 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c314103-3a39-445b-8440-984299190fae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:16.894424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c314103-3a39-445b-8440-984299190fae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:16.895968 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c314103-3a39-445b-8440-984299190fae None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:16.900380 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:16.914522 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 241/475] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:16.929449 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c314103-3a39-445b-8440-984299190fae None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:16.951322 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 235/476] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:16.980093 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521669be-58ee-4a3b-ba76-5ceb2fda7efd None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:17.072666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:17.105940 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c314103-3a39-445b-8440-984299190fae None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:17.121985 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521669be-58ee-4a3b-ba76-5ceb2fda7efd None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:17.149857 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3729c-a281-4220-a7b4-55c73fe703c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:17.150895 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3729c-a281-4220-a7b4-55c73fe703c1 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:17.151373 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3729c-a281-4220-a7b4-55c73fe703c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:17.151844 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3729c-a281-4220-a7b4-55c73fe703c1 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:17.152799 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba3729c-a281-4220-a7b4-55c73fe703c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:17.154438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba3729c-a281-4220-a7b4-55c73fe703c1 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:17.163326 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20941c12-6542-4303-8d98-2c7710cfeb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:17.164412 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20941c12-6542-4303-8d98-2c7710cfeb9c None admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:17.164788 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20941c12-6542-4303-8d98-2c7710cfeb9c None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:17.165073 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20941c12-6542-4303-8d98-2c7710cfeb9c None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:17.166203 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20941c12-6542-4303-8d98-2c7710cfeb9c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:17.168048 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20941c12-6542-4303-8d98-2c7710cfeb9c None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:17.173825 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.sql.core [None req-20941c12-6542-4303-8d98-2c7710cfeb9c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 23 12:19:17.173825 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 23 12:19:17.173825 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [parameters: {'id': 'c65372d6fe4e425b88f207df423d3716', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 23 12:19:17.173825 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80216) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 23 12:19:17.178524 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 242/477] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 12:19:16 2022] POST /identity/v3/users => generated 249 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:19:17.190966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-20941c12-6542-4303-8d98-2c7710cfeb9c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 23 12:19:17.190966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:17.190966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 12:19:17.190966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 12:19:17.190966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 12:19:17.190966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 12:19:17.190966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 12:19:17.190966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application result = self._query(query) May 23 12:19:17.190966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 12:19:17.190966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application conn.query(q) May 23 12:19:17.190966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 12:19:17.190966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 12:19:17.190966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 12:19:17.190966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application result.read() May 23 12:19:17.190966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 12:19:17.192196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 12:19:17.192196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 12:19:17.192196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 12:19:17.192196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 12:19:17.192196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 12:19:17.192196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 12:19:17.192196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 12:19:17.192196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 23 12:19:17.192196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:17.192196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 23 12:19:17.192196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:17.192196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:17.192196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 23 12:19:17.192196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 23 12:19:17.192196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 23 12:19:17.192196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return ref.to_dict() May 23 12:19:17.192196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 12:19:17.192196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application next(self.gen) May 23 12:19:17.194546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 23 12:19:17.194546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application yield resource May 23 12:19:17.194546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 12:19:17.194546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application next(self.gen) May 23 12:19:17.194546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 23 12:19:17.194546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application self.session.rollback() May 23 12:19:17.194546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 23 12:19:17.194546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application self.force_reraise() May 23 12:19:17.194546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 23 12:19:17.194546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise self.value May 23 12:19:17.194546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 23 12:19:17.194546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 23 12:19:17.194546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 23 12:19:17.194546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application session.commit() May 23 12:19:17.194546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 23 12:19:17.194546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 23 12:19:17.194546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 23 12:19:17.195685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application self._prepare_impl() May 23 12:19:17.195685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 23 12:19:17.195685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application self.session.flush() May 23 12:19:17.195685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 23 12:19:17.195685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application self._flush(objects) May 23 12:19:17.195685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 23 12:19:17.195685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 23 12:19:17.195685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 23 12:19:17.195685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application compat.raise_( May 23 12:19:17.195685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 12:19:17.195685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception May 23 12:19:17.195685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 23 12:19:17.195685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application flush_context.execute() May 23 12:19:17.195685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 23 12:19:17.195685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rec.execute(self) May 23 12:19:17.195685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 23 12:19:17.195685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 23 12:19:17.196618 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 23 12:19:17.196618 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application _emit_insert_statements( May 23 12:19:17.196618 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 23 12:19:17.196618 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application c = connection._execute_20( May 23 12:19:17.196618 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 23 12:19:17.196618 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 23 12:19:17.196618 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 23 12:19:17.196618 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 23 12:19:17.196618 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 23 12:19:17.196618 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ret = self._execute_context( May 23 12:19:17.196618 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 23 12:19:17.196618 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 23 12:19:17.196618 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 23 12:19:17.196618 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 23 12:19:17.196618 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 12:19:17.196618 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception May 23 12:19:17.197436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 12:19:17.197436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 12:19:17.197436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 12:19:17.197436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 12:19:17.197436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 12:19:17.197436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application result = self._query(query) May 23 12:19:17.197436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 12:19:17.197436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application conn.query(q) May 23 12:19:17.197436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 12:19:17.197436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 12:19:17.197436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 12:19:17.197436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application result.read() May 23 12:19:17.197436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 12:19:17.197436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 12:19:17.197436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 12:19:17.197436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 12:19:17.198381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 12:19:17.198381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 12:19:17.198381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 12:19:17.198381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 12:19:17.198381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 23 12:19:17.198381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 23 12:19:17.198381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [parameters: {'id': 'c65372d6fe4e425b88f207df423d3716', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 23 12:19:17.198381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 23 12:19:17.198381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:17.198381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:19:17.198381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:17.198381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:17.198381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:17.198381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:17.198381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:17.198381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:17.199427 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:17.199427 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:17.199427 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:17.199427 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:17.199427 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:17.199427 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:17.199427 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 23 12:19:17.199427 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 23 12:19:17.199427 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:17.199427 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:17.199427 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 23 12:19:17.199427 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 23 12:19:17.199427 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 23 12:19:17.199427 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 23 12:19:17.199427 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 23 12:19:17.199427 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:17.201270 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 236/478] 10.222.0.16 () {60 vars in 1131 bytes} [Mon May 23 12:19:16 2022] POST /identity/v3/roles => generated 144 bytes in 213 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 23 12:19:17.202643 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3729c-a281-4220-a7b4-55c73fe703c1 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:17.323004 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3729c-a281-4220-a7b4-55c73fe703c1 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:17.361052 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b926ef-7343-4c72-9188-8af9264b82dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:17.362588 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b926ef-7343-4c72-9188-8af9264b82dc None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:17.363011 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b926ef-7343-4c72-9188-8af9264b82dc None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:17.363427 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b926ef-7343-4c72-9188-8af9264b82dc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:17.374652 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b926ef-7343-4c72-9188-8af9264b82dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:17.384661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-89b926ef-7343-4c72-9188-8af9264b82dc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 12:19:17.384661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:17.384661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:17.384661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:17.384661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:17.384661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:17.384661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:17.384661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:17.384661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:17.384661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:17.384661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:17.384661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:17.384661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:17.384661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:17.384661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:17.384661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:17.385828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:17.385828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:17.385828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:17.385828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:17.385828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:17.385828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:17.385828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:17.385828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:17.385828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:17.385828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:17.385828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:17.385828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:17.385828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:17.385828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:17.385828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:17.385828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:17.386808 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:17.386808 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:17.386808 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:17.386808 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:17.386808 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:17.386808 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:17.386808 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:17.386808 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:17.386808 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:17.386808 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:17.386808 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 12:19:17.386808 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:17.389363 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 243/479] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 12:19:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:17.394888 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20941c12-6542-4303-8d98-2c7710cfeb9c None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:17.479123 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:17.548032 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20941c12-6542-4303-8d98-2c7710cfeb9c None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:17.593882 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff552fd-87a7-4b4d-8d8f-da59ac7b1722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:17.594485 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff552fd-87a7-4b4d-8d8f-da59ac7b1722 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:17.594774 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff552fd-87a7-4b4d-8d8f-da59ac7b1722 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:17.595053 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff552fd-87a7-4b4d-8d8f-da59ac7b1722 None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:17.595812 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff552fd-87a7-4b4d-8d8f-da59ac7b1722 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:17.596125 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff552fd-87a7-4b4d-8d8f-da59ac7b1722 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:17.597581 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff552fd-87a7-4b4d-8d8f-da59ac7b1722 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:17.612073 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 237/480] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:18.846526 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bf69e-03bf-4f53-ba50-c2607d678808 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:18.847070 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bf69e-03bf-4f53-ba50-c2607d678808 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:18.847070 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bf69e-03bf-4f53-ba50-c2607d678808 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:18.848770 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 244/481] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:18.854144 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138914e-0bfb-475e-9567-149e935755e7 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:18.854568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138914e-0bfb-475e-9567-149e935755e7 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:18.854781 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138914e-0bfb-475e-9567-149e935755e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:18.881922 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-f138914e-0bfb-475e-9567-149e935755e7 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:18.883076 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-f138914e-0bfb-475e-9567-149e935755e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:18.979565 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 238/482] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:19.033255 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9bf69e-03bf-4f53-ba50-c2607d678808 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:19.033430 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:19.174705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9bf69e-03bf-4f53-ba50-c2607d678808 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:19.226002 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701fbd44-d932-4b59-87ec-bdec09682029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:19.226817 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fbd44-d932-4b59-87ec-bdec09682029 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:19.227129 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fbd44-d932-4b59-87ec-bdec09682029 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:19.227415 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fbd44-d932-4b59-87ec-bdec09682029 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:19.239891 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701fbd44-d932-4b59-87ec-bdec09682029 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:19.248436 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-701fbd44-d932-4b59-87ec-bdec09682029 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:19.248436 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:19.248436 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:19.248436 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:19.248436 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:19.248436 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:19.248436 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:19.248436 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:19.248436 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:19.248436 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:19.248436 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:19.248436 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:19.248436 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:19.248436 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:19.248436 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:19.248436 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:19.249324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:19.249324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:19.249324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:19.249324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:19.249324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:19.249324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:19.249324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:19.249324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:19.249324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:19.249324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:19.249324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:19.249324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:19.249324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:19.249324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:19.249324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:19.249324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:19.250261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:19.250261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:19.250261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:19.250261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:19.250261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:19.250261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:19.250261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:19.250261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:19.250261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:19.250261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:19.250261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:19.250261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:19.250261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 245/483] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 12:19:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:19.261316 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138914e-0bfb-475e-9567-149e935755e7 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:19.321249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:19.368676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3ec10-e6cb-45c0-a3a7-917d1b8cccb8 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:19.369012 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3ec10-e6cb-45c0-a3a7-917d1b8cccb8 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:19.369325 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3ec10-e6cb-45c0-a3a7-917d1b8cccb8 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:19.370956 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 246/484] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:19:19.383698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51dfc38-f0a0-4e43-bea4-db6fef12a6d6 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:19.383989 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51dfc38-f0a0-4e43-bea4-db6fef12a6d6 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:19.384301 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51dfc38-f0a0-4e43-bea4-db6fef12a6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:19.384804 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138914e-0bfb-475e-9567-149e935755e7 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:19.409968 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a51dfc38-f0a0-4e43-bea4-db6fef12a6d6 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:19.411193 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a51dfc38-f0a0-4e43-bea4-db6fef12a6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:19.460143 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad019b-2cd7-4b06-ad3a-c542c6466d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:19.460541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad019b-2cd7-4b06-ad3a-c542c6466d6a None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:19.460827 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad019b-2cd7-4b06-ad3a-c542c6466d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:19.461106 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad019b-2cd7-4b06-ad3a-c542c6466d6a None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:19.461864 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ad019b-2cd7-4b06-ad3a-c542c6466d6a None admin] RBAC: Adding query filter params (name=service) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:19.462175 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ad019b-2cd7-4b06-ad3a-c542c6466d6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:19.463563 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ad019b-2cd7-4b06-ad3a-c542c6466d6a None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:19.473446 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 239/485] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 12:19:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:19.480348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad019b-2cd7-4b06-ad3a-c542c6466d6a None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:19.560549 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:19.596307 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ad019b-2cd7-4b06-ad3a-c542c6466d6a None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:19.602150 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 247/486] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 12:19:19.616688 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:19.645250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51dfc38-f0a0-4e43-bea4-db6fef12a6d6 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:19.656678 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92dca56-273e-4dab-b05c-104b30666e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:19.657498 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92dca56-273e-4dab-b05c-104b30666e9d None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:19.657787 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92dca56-273e-4dab-b05c-104b30666e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:19.658068 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92dca56-273e-4dab-b05c-104b30666e9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:19.672903 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92dca56-273e-4dab-b05c-104b30666e9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:19.674634 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92dca56-273e-4dab-b05c-104b30666e9d None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:19.695820 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-d92dca56-273e-4dab-b05c-104b30666e9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:19.695820 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:19.695820 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:19.695820 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:19.695820 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:19.695820 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:19.695820 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:19.695820 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:19.695820 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:19.695820 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:19.695820 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:19.695820 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:19.695820 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:19.695820 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:19.695820 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:19.696778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:19.696778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:19.696778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:19.696778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:19.696778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:19.696778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:19.696778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:19.696778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:19.696778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:19.696778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:19.696778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:19.696778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:19.696778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:19.696778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:19.696778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:19.696778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:19.696778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:19.698159 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:19.698159 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:19.698159 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:19.698159 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:19.698159 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 240/487] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:19:19.702887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92dca56-273e-4dab-b05c-104b30666e9d None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:19.814538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92dca56-273e-4dab-b05c-104b30666e9d None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:19.874304 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51dfc38-f0a0-4e43-bea4-db6fef12a6d6 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:19.881219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be830ac-84aa-482e-8102-1087ab2bcfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:19.881891 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be830ac-84aa-482e-8102-1087ab2bcfe4 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:19.882181 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be830ac-84aa-482e-8102-1087ab2bcfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:19.882585 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be830ac-84aa-482e-8102-1087ab2bcfe4 None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:19.883390 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be830ac-84aa-482e-8102-1087ab2bcfe4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:19.883705 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be830ac-84aa-482e-8102-1087ab2bcfe4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:19.885199 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be830ac-84aa-482e-8102-1087ab2bcfe4 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:19.904385 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 241/488] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:19.907136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38da718-e896-47e2-b9fa-e0e97e47cf4a None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:19.907430 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38da718-e896-47e2-b9fa-e0e97e47cf4a None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:19.907710 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38da718-e896-47e2-b9fa-e0e97e47cf4a None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:19.909244 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 242/489] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:19.910023 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38da718-e896-47e2-b9fa-e0e97e47cf4a None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:19.928703 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8450702d-bd4a-4d2a-b6ea-e33e5bd60ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:19.929516 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450702d-bd4a-4d2a-b6ea-e33e5bd60ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:19.929799 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450702d-bd4a-4d2a-b6ea-e33e5bd60ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:19.930074 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8450702d-bd4a-4d2a-b6ea-e33e5bd60ec5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:19.937841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:19.942200 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8450702d-bd4a-4d2a-b6ea-e33e5bd60ec5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:19.950285 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-8450702d-bd4a-4d2a-b6ea-e33e5bd60ec5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:19.950285 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:19.950285 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:19.950285 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:19.950285 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:19.950285 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:19.950285 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:19.950285 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:19.950285 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:19.950285 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:19.950285 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:19.950285 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:19.950285 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:19.950285 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:19.950285 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:19.950285 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:19.951172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:19.951172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:19.951172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:19.951172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:19.951172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:19.951172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:19.951172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:19.951172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:19.951172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:19.951172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:19.951172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:19.951172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:19.951172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:19.951172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:19.951172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:19.951172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:19.952807 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:19.952807 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:19.952807 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:19.952807 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:19.952807 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:19.952807 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:19.952807 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:19.952807 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:19.952807 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:19.952807 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:19.952807 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:19.952807 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:19.952807 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 248/490] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 12:19:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:19.954364 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290604b-aab1-4828-9963-2bef658e2255 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:19.954364 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290604b-aab1-4828-9963-2bef658e2255 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:19.954364 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290604b-aab1-4828-9963-2bef658e2255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:19.971820 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-2290604b-aab1-4828-9963-2bef658e2255 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:19.973036 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-2290604b-aab1-4828-9963-2bef658e2255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:20.036207 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38da718-e896-47e2-b9fa-e0e97e47cf4a None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:20.039275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:20.070976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 249/491] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:20.071912 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290604b-aab1-4828-9963-2bef658e2255 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:20.100201 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab7fe8-314f-441b-988c-3612b02e3151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:20.101013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab7fe8-314f-441b-988c-3612b02e3151 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:20.101299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab7fe8-314f-441b-988c-3612b02e3151 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:20.101576 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab7fe8-314f-441b-988c-3612b02e3151 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:20.115896 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab7fe8-314f-441b-988c-3612b02e3151 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:20.117462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab7fe8-314f-441b-988c-3612b02e3151 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:20.136116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-5dab7fe8-314f-441b-988c-3612b02e3151 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 23 12:19:20.136116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:20.136116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:20.136116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:20.136116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:20.136116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:20.136116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:20.136116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:20.136116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:20.136116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:20.136116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:20.136116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:20.136116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:19:20.136116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:19:20.136116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:19:20.136116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:19:20.137118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:20.137118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:20.137118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:19:20.137118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:20.137118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:19:20.137118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:20.137118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:20.137118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:20.137118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:20.137118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:20.137118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:20.137118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:20.137118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:20.137118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:20.137118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:20.137118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:20.137118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:20.137118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:20.138653 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:20.138653 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:20.138653 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:19:20.138653 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:19:20.138653 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:19:20.138653 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:19:20.138653 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:19:20.138653 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:19:20.138653 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 23 12:19:20.138653 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:20.138653 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 243/492] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:19:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:20.138653 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab7fe8-314f-441b-988c-3612b02e3151 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:20.206333 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290604b-aab1-4828-9963-2bef658e2255 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:20.228622 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:20.252471 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab7fe8-314f-441b-988c-3612b02e3151 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:20.254841 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a143d0-b909-4fff-a158-dc697408b65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:20.257056 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a143d0-b909-4fff-a158-dc697408b65e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:20.257338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a143d0-b909-4fff-a158-dc697408b65e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:20.260315 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a143d0-b909-4fff-a158-dc697408b65e None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:20.261289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a143d0-b909-4fff-a158-dc697408b65e None admin] RBAC: Adding query filter params (name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:20.261794 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a143d0-b909-4fff-a158-dc697408b65e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:20.263878 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a143d0-b909-4fff-a158-dc697408b65e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:20.280869 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 250/493] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 12:19:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:20.282338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a143d0-b909-4fff-a158-dc697408b65e None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:20.296746 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622840ab-7a22-4773-89b7-a39079f5b3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:20.297540 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622840ab-7a22-4773-89b7-a39079f5b3da None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:20.297825 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622840ab-7a22-4773-89b7-a39079f5b3da None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:20.298109 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622840ab-7a22-4773-89b7-a39079f5b3da None admin] PATH_INFO: `/v3/roles/service` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:20.309253 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622840ab-7a22-4773-89b7-a39079f5b3da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:20.317552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-622840ab-7a22-4773-89b7-a39079f5b3da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:20.317552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:20.317552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:20.317552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:20.317552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:20.317552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:20.317552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:20.317552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:20.317552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:20.317552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:20.317552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:20.317552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:20.317552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:20.317552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:20.317552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:20.317552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:20.319204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:20.319204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:20.319204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:20.319204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:20.319204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:20.319204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:20.319204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:20.319204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:20.319204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:20.319204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:20.319204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:20.319204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:20.319204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:20.319204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:20.319204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:20.319204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:20.320370 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:20.320370 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:20.320370 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:20.320370 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:20.320370 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:20.320370 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:20.320370 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:20.320370 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:20.320370 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:20.320370 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:20.320370 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:20.320370 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:20.320370 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 244/494] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 12:19:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:20.320370 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622840ab-7a22-4773-89b7-a39079f5b3da None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:20.345786 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:20.399147 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a143d0-b909-4fff-a158-dc697408b65e None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:20.434717 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622840ab-7a22-4773-89b7-a39079f5b3da None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:20.437024 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d53ed3d-a35a-4b32-9736-8449ac058885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:20.437713 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d53ed3d-a35a-4b32-9736-8449ac058885 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:20.438004 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d53ed3d-a35a-4b32-9736-8449ac058885 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:20.438285 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d53ed3d-a35a-4b32-9736-8449ac058885 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:20.439131 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d53ed3d-a35a-4b32-9736-8449ac058885 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:20.439444 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d53ed3d-a35a-4b32-9736-8449ac058885 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:20.440919 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d53ed3d-a35a-4b32-9736-8449ac058885 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:20.471484 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50b3d4-762c-418c-b5ca-f147135efc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:20.472308 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50b3d4-762c-418c-b5ca-f147135efc7f None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:20.472599 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50b3d4-762c-418c-b5ca-f147135efc7f None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:20.472878 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50b3d4-762c-418c-b5ca-f147135efc7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:20.485134 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 251/495] 10.222.0.16 () {56 vars in 1137 bytes} [Mon May 23 12:19:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:20.485134 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8936e47-3ed1-4283-9117-b58d4bb6b7e6 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:20.485134 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8936e47-3ed1-4283-9117-b58d4bb6b7e6 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:20.485134 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8936e47-3ed1-4283-9117-b58d4bb6b7e6 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:20.485134 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 252/496] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:20.485134 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8936e47-3ed1-4283-9117-b58d4bb6b7e6 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:20.486637 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:20.490462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50b3d4-762c-418c-b5ca-f147135efc7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:20.492356 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50b3d4-762c-418c-b5ca-f147135efc7f None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:20.510358 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-4c50b3d4-762c-418c-b5ca-f147135efc7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:20.510358 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:20.510358 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:20.510358 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:20.510358 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:20.510358 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:20.510358 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:20.510358 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:20.510358 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:20.510358 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:20.510358 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:20.510358 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:20.510358 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:20.510358 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:20.510358 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:20.511352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:20.511352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:20.511352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:20.511352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:20.511352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:20.511352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:20.511352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:20.511352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:20.511352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:20.511352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:20.511352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:20.511352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:20.511352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:20.511352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:20.511352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:20.511352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:20.511352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:20.512248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:20.512248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:20.512248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:20.512248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:20.512248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 245/497] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:20.512533 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50b3d4-762c-418c-b5ca-f147135efc7f None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:20.601311 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8936e47-3ed1-4283-9117-b58d4bb6b7e6 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:20.602896 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:20.619839 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50b3d4-762c-418c-b5ca-f147135efc7f None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:20.636628 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0f37d-fe38-4d0a-9cfe-2da873208138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:20.637291 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0f37d-fe38-4d0a-9cfe-2da873208138 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:20.637571 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0f37d-fe38-4d0a-9cfe-2da873208138 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:20.637849 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0f37d-fe38-4d0a-9cfe-2da873208138 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:20.638592 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca0f37d-fe38-4d0a-9cfe-2da873208138 None admin] RBAC: Adding query filter params (name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:20.638898 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca0f37d-fe38-4d0a-9cfe-2da873208138 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:20.640297 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca0f37d-fe38-4d0a-9cfe-2da873208138 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:20.648509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 253/498] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 12:19:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:20.649674 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f84b5-2d1d-4443-9c78-f417b809c306 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:20.649963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f84b5-2d1d-4443-9c78-f417b809c306 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:20.650244 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f84b5-2d1d-4443-9c78-f417b809c306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:20.657635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd3ddd-8222-43a2-ba81-1d211585b368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:20.658279 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd3ddd-8222-43a2-ba81-1d211585b368 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:20.658564 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd3ddd-8222-43a2-ba81-1d211585b368 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:20.658843 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd3ddd-8222-43a2-ba81-1d211585b368 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:20.667000 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-bd6f84b5-2d1d-4443-9c78-f417b809c306 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:20.668201 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-bd6f84b5-2d1d-4443-9c78-f417b809c306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:20.717277 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:20.727656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fd3ddd-8222-43a2-ba81-1d211585b368 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:20.729377 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fd3ddd-8222-43a2-ba81-1d211585b368 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:20.732528 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 254/499] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:20.733352 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f84b5-2d1d-4443-9c78-f417b809c306 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:20.827743 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:20.835064 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f84b5-2d1d-4443-9c78-f417b809c306 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:20.847649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 246/500] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:19:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:19:20.850308 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b83d06-ba07-43d7-a4aa-f06dfbe08840 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:20.850814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b83d06-ba07-43d7-a4aa-f06dfbe08840 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:20.851243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b83d06-ba07-43d7-a4aa-f06dfbe08840 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:20.852868 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 247/501] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:20.853521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b83d06-ba07-43d7-a4aa-f06dfbe08840 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:20.877416 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d125456-288b-4cd0-90fa-87f97d894f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:20.878080 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d125456-288b-4cd0-90fa-87f97d894f9b None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:20.878388 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d125456-288b-4cd0-90fa-87f97d894f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:20.878666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d125456-288b-4cd0-90fa-87f97d894f9b None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:20.879385 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d125456-288b-4cd0-90fa-87f97d894f9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:20.879693 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d125456-288b-4cd0-90fa-87f97d894f9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:20.881342 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d125456-288b-4cd0-90fa-87f97d894f9b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:20.892689 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 255/502] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:20.893466 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d125456-288b-4cd0-90fa-87f97d894f9b None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:20.947946 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:20.957641 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b83d06-ba07-43d7-a4aa-f06dfbe08840 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:20.992022 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec2c70-7fb1-4abd-a546-3abcfde4da97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:20.992833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec2c70-7fb1-4abd-a546-3abcfde4da97 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:20.993111 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec2c70-7fb1-4abd-a546-3abcfde4da97 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:20.993382 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec2c70-7fb1-4abd-a546-3abcfde4da97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:20.993706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d125456-288b-4cd0-90fa-87f97d894f9b None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:21.003185 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec2c70-7fb1-4abd-a546-3abcfde4da97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:21.004661 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec2c70-7fb1-4abd-a546-3abcfde4da97 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:21.022329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-42ec2c70-7fb1-4abd-a546-3abcfde4da97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:21.022329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:21.022329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:21.022329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:21.022329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:21.022329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:21.022329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:21.022329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:21.022329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:21.022329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:21.022329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:21.022329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:21.022329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:21.022329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:21.022329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:21.023359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:21.023359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:21.023359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:21.023359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:21.023359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:21.023359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:21.023359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:21.023359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:21.023359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:21.023359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:21.023359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:21.023359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:21.023359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:21.023359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:21.023359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:21.023359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:21.023359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:21.024296 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:21.024296 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:21.024296 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:21.024296 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:21.024296 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 248/503] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:21.024662 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec2c70-7fb1-4abd-a546-3abcfde4da97 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:21.031945 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9e6a7-a799-4d70-87ff-813c18b1af59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:21.032767 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9e6a7-a799-4d70-87ff-813c18b1af59 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:21.033050 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9e6a7-a799-4d70-87ff-813c18b1af59 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:21.033326 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9e6a7-a799-4d70-87ff-813c18b1af59 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:21.043995 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f9e6a7-a799-4d70-87ff-813c18b1af59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:21.051646 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-f9f9e6a7-a799-4d70-87ff-813c18b1af59 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:21.051646 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:21.051646 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:21.051646 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:21.051646 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:21.051646 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:21.051646 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:21.051646 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:21.051646 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:21.051646 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:21.051646 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:21.051646 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:21.051646 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:21.051646 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:21.051646 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:21.051646 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:21.053142 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:21.053142 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:21.053142 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:21.053142 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:21.053142 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:21.053142 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:21.053142 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:21.053142 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:21.053142 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:21.053142 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:21.053142 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:21.053142 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:21.053142 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:21.053142 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:21.053142 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:21.053142 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:21.053954 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:21.053954 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:21.053954 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:21.053954 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:21.053954 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:21.053954 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:21.053954 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:21.053954 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:21.053954 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:21.053954 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:21.053954 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:21.053954 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:21.053954 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 256/504] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 12:19:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:21.054802 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912931a-d435-4e5c-956b-4aef1a398d6e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:21.055025 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912931a-d435-4e5c-956b-4aef1a398d6e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:21.055376 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912931a-d435-4e5c-956b-4aef1a398d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:21.071518 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-c912931a-d435-4e5c-956b-4aef1a398d6e None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:21.072719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c912931a-d435-4e5c-956b-4aef1a398d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:21.081697 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:21.147471 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec2c70-7fb1-4abd-a546-3abcfde4da97 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:21.152781 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 257/505] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:21.154098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912931a-d435-4e5c-956b-4aef1a398d6e None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:21.200503 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22bc3b-0975-4df5-9472-ad571d26d65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:21.201283 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22bc3b-0975-4df5-9472-ad571d26d65a None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:21.201567 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22bc3b-0975-4df5-9472-ad571d26d65a None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:21.201843 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22bc3b-0975-4df5-9472-ad571d26d65a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:21.212522 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22bc3b-0975-4df5-9472-ad571d26d65a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:21.213971 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22bc3b-0975-4df5-9472-ad571d26d65a None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:21.216498 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:21.232312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-7a22bc3b-0975-4df5-9472-ad571d26d65a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:21.232312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:21.232312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:21.232312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:21.232312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:21.232312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:21.232312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:21.232312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:21.232312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:21.232312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:21.232312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:21.232312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:21.232312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:21.232312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:21.232312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:21.233214 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:21.233214 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:21.233214 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:21.233214 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:21.233214 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:21.233214 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:21.233214 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:21.233214 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:21.233214 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:21.233214 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:21.233214 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:21.233214 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:21.233214 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:21.233214 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:21.233214 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:21.233214 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:21.233214 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:21.234069 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:21.234069 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:21.234069 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:21.234069 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:21.234069 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 249/506] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:21.234345 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22bc3b-0975-4df5-9472-ad571d26d65a None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:21.258971 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912931a-d435-4e5c-956b-4aef1a398d6e None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:21.292419 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cab0e0-f583-49f6-a8c0-070e60212364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:21.293201 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cab0e0-f583-49f6-a8c0-070e60212364 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:21.293479 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cab0e0-f583-49f6-a8c0-070e60212364 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:21.293756 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cab0e0-f583-49f6-a8c0-070e60212364 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:21.300705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cab0e0-f583-49f6-a8c0-070e60212364 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:21.302154 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cab0e0-f583-49f6-a8c0-070e60212364 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:21.319095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-f5cab0e0-f583-49f6-a8c0-070e60212364 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 12:19:21.319095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:21.319095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:21.319095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:21.319095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:21.319095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:21.319095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:21.319095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:21.319095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:21.319095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:21.319095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:21.319095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:21.319095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:19:21.319095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:19:21.319095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:19:21.319095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:19:21.320215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:21.320215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:21.320215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:19:21.320215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:21.320215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:19:21.320215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:21.320215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:21.320215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:21.320215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:21.320215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:21.320215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:21.320215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:21.320215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:21.320215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:21.320215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:21.320215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:21.320215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:21.320215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:21.321124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:21.321124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:21.321124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:19:21.321124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:19:21.321124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:19:21.321124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:19:21.321124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:19:21.321124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:19:21.321124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 12:19:21.321124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:21.321124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 258/507] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:19:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:21.321124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cab0e0-f583-49f6-a8c0-070e60212364 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:21.325316 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:21.336749 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22bc3b-0975-4df5-9472-ad571d26d65a None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:21.370697 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c8ea1-95eb-4cd6-950b-b7715ab54e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:21.371416 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c8ea1-95eb-4cd6-950b-b7715ab54e02 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:21.371707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c8ea1-95eb-4cd6-950b-b7715ab54e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:21.371986 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c8ea1-95eb-4cd6-950b-b7715ab54e02 None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:21.372741 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833c8ea1-95eb-4cd6-950b-b7715ab54e02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:21.373052 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833c8ea1-95eb-4cd6-950b-b7715ab54e02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:21.374444 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833c8ea1-95eb-4cd6-950b-b7715ab54e02 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:21.386229 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 250/508] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:21.387052 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c8ea1-95eb-4cd6-950b-b7715ab54e02 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:21.427212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cab0e0-f583-49f6-a8c0-070e60212364 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:21.445617 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:21.463037 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85077e4a-4a0e-4559-9184-5b50cd707257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:21.463726 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85077e4a-4a0e-4559-9184-5b50cd707257 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:21.464014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85077e4a-4a0e-4559-9184-5b50cd707257 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:21.464330 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85077e4a-4a0e-4559-9184-5b50cd707257 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:21.465083 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85077e4a-4a0e-4559-9184-5b50cd707257 None admin] RBAC: Adding query filter params (name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:21.465397 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85077e4a-4a0e-4559-9184-5b50cd707257 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:21.466774 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85077e4a-4a0e-4559-9184-5b50cd707257 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:21.474705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 259/509] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 12:19:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:21.475483 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85077e4a-4a0e-4559-9184-5b50cd707257 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:21.495554 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c8ea1-95eb-4cd6-950b-b7715ab54e02 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:21.529185 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88db7f6-1612-4bee-9521-dde8bbfedfbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:21.529980 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88db7f6-1612-4bee-9521-dde8bbfedfbf None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:21.530269 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88db7f6-1612-4bee-9521-dde8bbfedfbf None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:21.530552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88db7f6-1612-4bee-9521-dde8bbfedfbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:21.538573 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:21.541149 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88db7f6-1612-4bee-9521-dde8bbfedfbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:21.542612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88db7f6-1612-4bee-9521-dde8bbfedfbf None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:21.560245 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-b88db7f6-1612-4bee-9521-dde8bbfedfbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:21.560245 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:21.560245 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:21.560245 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:21.560245 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:21.560245 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:21.560245 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:21.560245 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:21.560245 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:21.560245 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:21.560245 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:21.560245 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:21.560245 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:21.560245 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:21.560245 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:21.561219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:21.561219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:21.561219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:21.561219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:21.561219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:21.561219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:21.561219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:21.561219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:21.561219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:21.561219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:21.561219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:21.561219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:21.561219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:21.561219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:21.561219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:21.561219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:21.561219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:21.562095 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:21.562095 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:21.562095 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:21.562095 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:21.562095 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 251/510] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:21.562362 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88db7f6-1612-4bee-9521-dde8bbfedfbf None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:21.582861 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85077e4a-4a0e-4559-9184-5b50cd707257 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:21.617267 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e113db-cfe6-4feb-8a22-bf69d8d47445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:21.617931 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e113db-cfe6-4feb-8a22-bf69d8d47445 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:21.618220 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e113db-cfe6-4feb-8a22-bf69d8d47445 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:21.618499 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e113db-cfe6-4feb-8a22-bf69d8d47445 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:21.619217 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e113db-cfe6-4feb-8a22-bf69d8d47445 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:21.619530 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e113db-cfe6-4feb-8a22-bf69d8d47445 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:21.620920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e113db-cfe6-4feb-8a22-bf69d8d47445 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:21.632446 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 260/511] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:21.633244 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e113db-cfe6-4feb-8a22-bf69d8d47445 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:21.662661 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88db7f6-1612-4bee-9521-dde8bbfedfbf None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:21.671356 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:21.697453 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df0229-cd21-436c-b858-4f51978248ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:21.698057 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df0229-cd21-436c-b858-4f51978248ff None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:21.698349 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df0229-cd21-436c-b858-4f51978248ff None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:21.698641 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df0229-cd21-436c-b858-4f51978248ff None admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:21.699479 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df0229-cd21-436c-b858-4f51978248ff None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:21.699802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df0229-cd21-436c-b858-4f51978248ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:21.701282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df0229-cd21-436c-b858-4f51978248ff None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:21.732438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e113db-cfe6-4feb-8a22-bf69d8d47445 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:21.737534 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 252/512] 10.222.0.16 () {56 vars in 1137 bytes} [Mon May 23 12:19:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:21.738352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df0229-cd21-436c-b858-4f51978248ff None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:21.758472 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:21.767646 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa582e8-3a67-466e-8690-b85a187497f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:21.768473 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa582e8-3a67-466e-8690-b85a187497f7 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:21.768764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa582e8-3a67-466e-8690-b85a187497f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:21.769051 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa582e8-3a67-466e-8690-b85a187497f7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:21.776450 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa582e8-3a67-466e-8690-b85a187497f7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:21.777892 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa582e8-3a67-466e-8690-b85a187497f7 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:21.793212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-cfa582e8-3a67-466e-8690-b85a187497f7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 12:19:21.793212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:21.793212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:21.793212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:21.793212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:21.793212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:21.793212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:21.793212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:21.793212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:21.793212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:21.793212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:21.793212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:21.793212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:19:21.793212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:19:21.793212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:19:21.793212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:19:21.794127 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:21.794127 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:21.794127 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:19:21.794127 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:21.794127 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:19:21.794127 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:21.794127 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:21.794127 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:21.794127 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:21.794127 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:21.794127 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:21.794127 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:21.794127 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:21.794127 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:21.794127 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:21.794127 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:21.794127 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:21.794127 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:21.795009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:21.795009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:21.795009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:19:21.795009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:19:21.795009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:19:21.795009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:19:21.795009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:19:21.795009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:19:21.795009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 12:19:21.795009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:21.795009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 261/513] 10.222.0.16 () {56 vars in 1093 bytes} [Mon May 23 12:19:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:21.795557 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa582e8-3a67-466e-8690-b85a187497f7 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:21.841058 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df0229-cd21-436c-b858-4f51978248ff None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:21.875431 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355cc65a-4227-4b75-bac2-adefc87f0092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:21.876249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355cc65a-4227-4b75-bac2-adefc87f0092 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:21.876535 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355cc65a-4227-4b75-bac2-adefc87f0092 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:21.876815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355cc65a-4227-4b75-bac2-adefc87f0092 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:21.886746 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355cc65a-4227-4b75-bac2-adefc87f0092 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:21.888248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355cc65a-4227-4b75-bac2-adefc87f0092 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:21.893470 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa582e8-3a67-466e-8690-b85a187497f7 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:21.905390 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-355cc65a-4227-4b75-bac2-adefc87f0092 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:21.905390 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:21.905390 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:21.905390 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:21.905390 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:21.905390 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:21.905390 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:21.905390 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:21.905390 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:21.905390 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:21.905390 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:21.905390 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:21.905390 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:21.905390 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:21.905390 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:21.906395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:21.906395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:21.906395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:21.906395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:21.906395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:21.906395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:21.906395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:21.906395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:21.906395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:21.906395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:21.906395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:21.906395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:21.906395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:21.906395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:21.906395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:21.906395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:21.906395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:21.907589 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:21.907589 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:21.907589 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:21.907589 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:21.907589 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 253/514] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:21.907589 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355cc65a-4227-4b75-bac2-adefc87f0092 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:21.929532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:21.930959 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f409ade-8225-4cf0-a167-ea6600c303f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:21.931631 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f409ade-8225-4cf0-a167-ea6600c303f6 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:21.931917 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f409ade-8225-4cf0-a167-ea6600c303f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:21.932224 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f409ade-8225-4cf0-a167-ea6600c303f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:21.932949 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f409ade-8225-4cf0-a167-ea6600c303f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:21.933261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f409ade-8225-4cf0-a167-ea6600c303f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:21.934666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f409ade-8225-4cf0-a167-ea6600c303f6 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:21.946419 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 262/515] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:21.947183 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f409ade-8225-4cf0-a167-ea6600c303f6 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:21.994957 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:22.019633 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355cc65a-4227-4b75-bac2-adefc87f0092 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:22.049280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f409ade-8225-4cf0-a167-ea6600c303f6 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:22.054291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7739ceaf-e5dc-4e6d-a0a6-ca4cebd43b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:22.055081 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7739ceaf-e5dc-4e6d-a0a6-ca4cebd43b0a None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:22.055387 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7739ceaf-e5dc-4e6d-a0a6-ca4cebd43b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:22.055667 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7739ceaf-e5dc-4e6d-a0a6-ca4cebd43b0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:22.062697 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7739ceaf-e5dc-4e6d-a0a6-ca4cebd43b0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:22.064151 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7739ceaf-e5dc-4e6d-a0a6-ca4cebd43b0a None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:22.079215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-7739ceaf-e5dc-4e6d-a0a6-ca4cebd43b0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:22.079215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:22.079215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:22.079215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:22.079215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:22.079215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:22.079215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:22.079215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:22.079215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:22.079215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:22.079215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:22.079215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:22.079215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:19:22.079215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:19:22.079215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:19:22.080255 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:19:22.080255 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:22.080255 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:22.080255 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:22.080255 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:22.080255 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:22.080255 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:22.080255 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:22.080255 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:22.080255 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:22.080255 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:22.080255 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:22.080255 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:22.080255 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:22.080255 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:22.080255 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:22.080255 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:22.081165 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:19:22.081165 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:19:22.081165 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:19:22.081165 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:19:22.081165 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:19:22.081165 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:19:22.081165 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:22.081165 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:22.081165 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 254/516] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:22.081623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7739ceaf-e5dc-4e6d-a0a6-ca4cebd43b0a None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:22.084670 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f7f95-337a-44ef-9e06-9b7f4ac65a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:22.085320 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f7f95-337a-44ef-9e06-9b7f4ac65a1d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:22.085777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f7f95-337a-44ef-9e06-9b7f4ac65a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:22.085882 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f7f95-337a-44ef-9e06-9b7f4ac65a1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:22.151612 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f7f95-337a-44ef-9e06-9b7f4ac65a1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:22.153110 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f7f95-337a-44ef-9e06-9b7f4ac65a1d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:22.158616 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:22.181228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7739ceaf-e5dc-4e6d-a0a6-ca4cebd43b0a None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:22.199212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:22.216223 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65aa9b3-c6f7-47ec-b9bf-92d9ecc115d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:22.216890 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65aa9b3-c6f7-47ec-b9bf-92d9ecc115d8 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:22.217173 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65aa9b3-c6f7-47ec-b9bf-92d9ecc115d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:22.217450 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65aa9b3-c6f7-47ec-b9bf-92d9ecc115d8 None admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:22.218277 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65aa9b3-c6f7-47ec-b9bf-92d9ecc115d8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:22.218590 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65aa9b3-c6f7-47ec-b9bf-92d9ecc115d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:22.220032 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65aa9b3-c6f7-47ec-b9bf-92d9ecc115d8 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:22.238512 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 255/517] 10.222.0.16 () {56 vars in 1133 bytes} [Mon May 23 12:19:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:22.239318 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65aa9b3-c6f7-47ec-b9bf-92d9ecc115d8 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:22.258164 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 263/518] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:19:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:19:22.258906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f7f95-337a-44ef-9e06-9b7f4ac65a1d None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:22.343927 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65aa9b3-c6f7-47ec-b9bf-92d9ecc115d8 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:22.356616 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f7f95-337a-44ef-9e06-9b7f4ac65a1d None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:22.377415 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0ef42-7fdb-4e36-873b-616374bce71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:22.378073 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0ef42-7fdb-4e36-873b-616374bce71e None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:22.378363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0ef42-7fdb-4e36-873b-616374bce71e None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:22.378714 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0ef42-7fdb-4e36-873b-616374bce71e None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:22.379573 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0ef42-7fdb-4e36-873b-616374bce71e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:22.379680 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0ef42-7fdb-4e36-873b-616374bce71e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:22.381794 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0ef42-7fdb-4e36-873b-616374bce71e None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:22.384251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:22.390837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b4134d-c9b8-47ba-ae5c-eeaabd039837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:22.391506 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4134d-c9b8-47ba-ae5c-eeaabd039837 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:22.391787 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4134d-c9b8-47ba-ae5c-eeaabd039837 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:22.392066 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4134d-c9b8-47ba-ae5c-eeaabd039837 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:22.392796 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b4134d-c9b8-47ba-ae5c-eeaabd039837 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:22.393888 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 256/519] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:22.394232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b4134d-c9b8-47ba-ae5c-eeaabd039837 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:22.394688 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0ef42-7fdb-4e36-873b-616374bce71e None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:22.414137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 264/520] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 12:19:22 2022] POST /identity/v3/users => generated 247 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:19:22.414979 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b4134d-c9b8-47ba-ae5c-eeaabd039837 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:22.417596 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:22.525816 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0ef42-7fdb-4e36-873b-616374bce71e None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:22.535046 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b4134d-c9b8-47ba-ae5c-eeaabd039837 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:22.564068 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c72f4d-3c74-4c43-85f9-6054e69ec4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:22.564861 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c72f4d-3c74-4c43-85f9-6054e69ec4bd None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:22.565149 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c72f4d-3c74-4c43-85f9-6054e69ec4bd None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:22.565434 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c72f4d-3c74-4c43-85f9-6054e69ec4bd None admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:22.566224 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c72f4d-3c74-4c43-85f9-6054e69ec4bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:22.566537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c72f4d-3c74-4c43-85f9-6054e69ec4bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:22.568827 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c72f4d-3c74-4c43-85f9-6054e69ec4bd None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:22.573391 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71914ada-d828-49a9-93d6-986aa940cee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:22.574060 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71914ada-d828-49a9-93d6-986aa940cee2 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:22.574347 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71914ada-d828-49a9-93d6-986aa940cee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:22.574628 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71914ada-d828-49a9-93d6-986aa940cee2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:22.591659 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 257/521] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:19:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:22.592850 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c72f4d-3c74-4c43-85f9-6054e69ec4bd None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:22.659282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:22.666551 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71914ada-d828-49a9-93d6-986aa940cee2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:22.668003 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71914ada-d828-49a9-93d6-986aa940cee2 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:22.670021 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:22.709775 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c72f4d-3c74-4c43-85f9-6054e69ec4bd None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:22.744735 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76baf02-8863-4c61-bca1-4f24de4edbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:22.745536 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76baf02-8863-4c61-bca1-4f24de4edbe5 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:22.745823 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76baf02-8863-4c61-bca1-4f24de4edbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:22.746102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76baf02-8863-4c61-bca1-4f24de4edbe5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:22.757184 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76baf02-8863-4c61-bca1-4f24de4edbe5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:22.759159 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76baf02-8863-4c61-bca1-4f24de4edbe5 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:22.779545 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-a76baf02-8863-4c61-bca1-4f24de4edbe5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:22.779545 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:22.779545 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:22.779545 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:22.779545 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:22.779545 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:22.779545 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:22.779545 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:22.779545 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:22.779545 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:22.779545 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:22.779545 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:22.779545 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:22.779545 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:22.779545 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:22.781522 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:22.781522 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:22.781522 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:22.781522 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:22.781522 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:22.781522 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:22.781522 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:22.781522 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:22.781522 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:22.781522 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:22.781522 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:22.781522 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:22.781522 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:22.781522 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:22.781522 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:22.781522 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:22.781522 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:22.782598 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:22.782598 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:22.782598 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:22.782598 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:22.782598 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 258/522] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:22.782598 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76baf02-8863-4c61-bca1-4f24de4edbe5 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:22.792344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 265/523] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:19:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:19:22.793425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71914ada-d828-49a9-93d6-986aa940cee2 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:22.889959 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76baf02-8863-4c61-bca1-4f24de4edbe5 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:22.899093 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:22.903525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71914ada-d828-49a9-93d6-986aa940cee2 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:22.905228 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:22.926525 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91504eef-6d6e-421a-a09f-8581589edc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:22.928308 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91504eef-6d6e-421a-a09f-8581589edc87 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:22.928606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91504eef-6d6e-421a-a09f-8581589edc87 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:22.928892 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91504eef-6d6e-421a-a09f-8581589edc87 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:22.937051 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91504eef-6d6e-421a-a09f-8581589edc87 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:22.938516 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91504eef-6d6e-421a-a09f-8581589edc87 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:22.945835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f7d6dd-8746-4152-adcf-01f72991b590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:22.946704 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f7d6dd-8746-4152-adcf-01f72991b590 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:22.946990 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f7d6dd-8746-4152-adcf-01f72991b590 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:22.947269 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f7d6dd-8746-4152-adcf-01f72991b590 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:22.948109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f7d6dd-8746-4152-adcf-01f72991b590 None admin] RBAC: Adding query filter params (role.id=81d82471e2be4f0aa8164b75c16a9718, scope.project.id=839585501f414e1ab02501422510e54f, user.id=2483a351f5e943ad8c872a98a4717b0b) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:22.948474 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f7d6dd-8746-4152-adcf-01f72991b590 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:22.949903 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f7d6dd-8746-4152-adcf-01f72991b590 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:22.954187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-91504eef-6d6e-421a-a09f-8581589edc87 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 23 12:19:22.954187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:22.954187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:22.954187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:22.954187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:22.954187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:22.954187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:22.954187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:22.954187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:22.954187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:22.954187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:22.954187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:22.954187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:19:22.954187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:19:22.954187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:19:22.954187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:19:22.955372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:22.955372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:22.955372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:19:22.955372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:22.955372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:19:22.955372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:22.955372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:22.955372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:22.955372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:22.955372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:22.955372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:22.955372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:22.955372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:22.955372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:22.955372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:22.955372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:22.955372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:22.955372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:22.956543 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:22.956543 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:22.956543 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:19:22.956543 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:19:22.956543 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:19:22.956543 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:19:22.956543 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:19:22.956543 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:19:22.956543 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 23 12:19:22.956543 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:22.958778 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 266/524] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:19:22 2022] GET /identity/v3/role_assignments?user.id=2483a351f5e943ad8c872a98a4717b0b&scope.project.id=839585501f414e1ab02501422510e54f&role.id=81d82471e2be4f0aa8164b75c16a9718 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:22.959798 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f7d6dd-8746-4152-adcf-01f72991b590 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:22.960428 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 259/525] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 12:19:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:22.961297 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91504eef-6d6e-421a-a09f-8581589edc87 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:23.103015 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91504eef-6d6e-421a-a09f-8581589edc87 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:23.111367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f7d6dd-8746-4152-adcf-01f72991b590 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:23.142016 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9492461-a59e-4093-8c86-6a0f3261bcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:23.142833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9492461-a59e-4093-8c86-6a0f3261bcee None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:23.143121 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9492461-a59e-4093-8c86-6a0f3261bcee None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:23.143400 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9492461-a59e-4093-8c86-6a0f3261bcee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:23.155484 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9492461-a59e-4093-8c86-6a0f3261bcee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:23.157080 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9492461-a59e-4093-8c86-6a0f3261bcee None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:23.158542 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb234be-6c92-44cd-b2db-4507a1589789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:23.159709 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb234be-6c92-44cd-b2db-4507a1589789 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:23.160662 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb234be-6c92-44cd-b2db-4507a1589789 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:23.161120 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb234be-6c92-44cd-b2db-4507a1589789 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:23.162733 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb234be-6c92-44cd-b2db-4507a1589789 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:23.163208 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb234be-6c92-44cd-b2db-4507a1589789 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:23.164808 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb234be-6c92-44cd-b2db-4507a1589789 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:23.176780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-e9492461-a59e-4093-8c86-6a0f3261bcee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:23.176780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:23.176780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:23.176780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:23.176780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:23.176780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:23.176780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:23.176780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:23.176780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:23.176780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:23.176780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:23.176780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:23.176780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:23.176780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:23.176780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:23.178531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:23.178531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:23.178531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:23.178531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:23.178531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:23.178531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:23.178531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:23.178531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:23.178531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:23.178531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:23.178531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:23.178531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:23.178531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:23.178531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:23.178531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:23.178531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:23.178531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:23.179805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:23.179805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:23.179805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:23.179805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:23.179805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 260/526] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:23.179805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9492461-a59e-4093-8c86-6a0f3261bcee None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:23.183254 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 267/527] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:23.187735 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb234be-6c92-44cd-b2db-4507a1589789 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:23.211922 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:23.221385 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:23.301429 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9492461-a59e-4093-8c86-6a0f3261bcee None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:23.302525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb234be-6c92-44cd-b2db-4507a1589789 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:23.337987 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6608157-ba04-4303-b87b-cffed776659d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:23.339062 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6608157-ba04-4303-b87b-cffed776659d None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:23.339426 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6608157-ba04-4303-b87b-cffed776659d None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:23.339554 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1190e1-9c57-4539-85a2-030bf627236c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:23.339777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6608157-ba04-4303-b87b-cffed776659d None admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:23.340318 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1190e1-9c57-4539-85a2-030bf627236c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:23.340614 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1190e1-9c57-4539-85a2-030bf627236c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:23.340844 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6608157-ba04-4303-b87b-cffed776659d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:23.340944 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1190e1-9c57-4539-85a2-030bf627236c None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:23.341289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6608157-ba04-4303-b87b-cffed776659d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:23.341608 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1190e1-9c57-4539-85a2-030bf627236c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:23.341923 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1190e1-9c57-4539-85a2-030bf627236c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:23.343039 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6608157-ba04-4303-b87b-cffed776659d None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:23.343292 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1190e1-9c57-4539-85a2-030bf627236c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:23.356005 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 268/528] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:23.356960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1190e1-9c57-4539-85a2-030bf627236c None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:23.362050 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 261/529] 10.222.0.16 () {56 vars in 1139 bytes} [Mon May 23 12:19:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:23.363067 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6608157-ba04-4303-b87b-cffed776659d None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:23.444139 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:23.464492 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1190e1-9c57-4539-85a2-030bf627236c None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:23.468297 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6608157-ba04-4303-b87b-cffed776659d None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:23.504409 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:23.504971 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63996438-50cb-4476-9b61-588d0c3c66a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:23.506604 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63996438-50cb-4476-9b61-588d0c3c66a8 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:23.507052 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cef896-e7a5-4510-89b3-6a7eeb48f14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:23.507222 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63996438-50cb-4476-9b61-588d0c3c66a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:23.507632 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63996438-50cb-4476-9b61-588d0c3c66a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:23.508324 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cef896-e7a5-4510-89b3-6a7eeb48f14b None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:23.508717 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cef896-e7a5-4510-89b3-6a7eeb48f14b None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:23.509075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cef896-e7a5-4510-89b3-6a7eeb48f14b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:23.516316 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cef896-e7a5-4510-89b3-6a7eeb48f14b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:23.516648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63996438-50cb-4476-9b61-588d0c3c66a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:23.518012 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cef896-e7a5-4510-89b3-6a7eeb48f14b None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:23.518416 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63996438-50cb-4476-9b61-588d0c3c66a8 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:23.532609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-60cef896-e7a5-4510-89b3-6a7eeb48f14b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:23.532609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:23.532609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:23.532609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:23.532609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:23.532609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:23.532609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:23.532609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:23.532609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:23.532609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:23.532609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:23.532609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:23.532609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:19:23.532609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:19:23.532609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:19:23.533495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:19:23.533495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:23.533495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:23.533495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:23.533495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:23.533495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:23.533495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:23.533495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:23.533495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:23.533495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:23.533495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:23.533495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:23.533495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:23.533495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:23.533495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:23.533495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:23.533495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:23.534355 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:19:23.534355 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:19:23.534355 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:19:23.534355 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:19:23.534355 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:19:23.534355 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:19:23.534355 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:23.534355 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:23.534355 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 262/530] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:23.534816 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cef896-e7a5-4510-89b3-6a7eeb48f14b None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:23.537674 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-63996438-50cb-4476-9b61-588d0c3c66a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:23.537674 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:23.537674 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:23.537674 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:23.537674 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:23.537674 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:23.537674 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:23.537674 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:23.537674 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:23.537674 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:23.537674 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:23.537674 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:23.537674 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:19:23.537674 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:19:23.537674 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:19:23.538548 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:19:23.538548 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:23.538548 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:23.538548 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:23.538548 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:23.538548 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:23.538548 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:23.538548 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:23.538548 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:23.538548 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:23.538548 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:23.538548 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:23.538548 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:23.538548 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:23.538548 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:23.538548 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:23.538548 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:23.539412 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:19:23.539412 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:19:23.539412 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:19:23.539412 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:19:23.539412 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:19:23.539412 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:19:23.539412 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:23.539412 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:23.541638 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 269/531] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:23.552288 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63996438-50cb-4476-9b61-588d0c3c66a8 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:23.665703 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cef896-e7a5-4510-89b3-6a7eeb48f14b None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:23.677786 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63996438-50cb-4476-9b61-588d0c3c66a8 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:23.711101 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5913fe3-4b9e-40d3-a2ba-7d51be78806b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:23.711767 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5913fe3-4b9e-40d3-a2ba-7d51be78806b None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:23.712053 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5913fe3-4b9e-40d3-a2ba-7d51be78806b None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:23.712368 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5913fe3-4b9e-40d3-a2ba-7d51be78806b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:23.714766 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0127ee-bb3a-4873-adcd-d53c3009c070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:23.715592 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0127ee-bb3a-4873-adcd-d53c3009c070 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:23.716024 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0127ee-bb3a-4873-adcd-d53c3009c070 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:23.716493 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0127ee-bb3a-4873-adcd-d53c3009c070 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:23.717430 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0127ee-bb3a-4873-adcd-d53c3009c070 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:23.717885 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0127ee-bb3a-4873-adcd-d53c3009c070 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:23.719464 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0127ee-bb3a-4873-adcd-d53c3009c070 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:23.721623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:23.745854 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 270/532] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:19:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:23.747102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0127ee-bb3a-4873-adcd-d53c3009c070 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:23.756577 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:23.794927 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5913fe3-4b9e-40d3-a2ba-7d51be78806b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:23.796803 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5913fe3-4b9e-40d3-a2ba-7d51be78806b None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:23.857739 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0127ee-bb3a-4873-adcd-d53c3009c070 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:23.894245 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5779f146-d00c-4223-8620-bb5fd37066a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:23.895256 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5779f146-d00c-4223-8620-bb5fd37066a8 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:23.895722 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5779f146-d00c-4223-8620-bb5fd37066a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:23.896828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5779f146-d00c-4223-8620-bb5fd37066a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:23.897998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5779f146-d00c-4223-8620-bb5fd37066a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:23.898552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5779f146-d00c-4223-8620-bb5fd37066a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:23.900199 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5779f146-d00c-4223-8620-bb5fd37066a8 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:23.918754 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 271/533] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:19:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:23.920313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5779f146-d00c-4223-8620-bb5fd37066a8 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:23.922709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 263/534] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:19:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:19:23.922709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5913fe3-4b9e-40d3-a2ba-7d51be78806b None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:23.971949 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:24.015148 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:24.047589 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5913fe3-4b9e-40d3-a2ba-7d51be78806b None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:24.057573 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5779f146-d00c-4223-8620-bb5fd37066a8 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:24.086560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadf1bc4-3f47-4bda-bd82-89737d7545ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:24.087690 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf1bc4-3f47-4bda-bd82-89737d7545ae None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:24.088311 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf1bc4-3f47-4bda-bd82-89737d7545ae None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:24.088841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf1bc4-3f47-4bda-bd82-89737d7545ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:24.090176 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf1bc4-3f47-4bda-bd82-89737d7545ae None admin] RBAC: Adding query filter params (role.id=81d82471e2be4f0aa8164b75c16a9718, scope.project.id=839585501f414e1ab02501422510e54f, user.id=b83535b443a84275a160cd84cb3ad5a0) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:24.090761 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf1bc4-3f47-4bda-bd82-89737d7545ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:24.092633 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf1bc4-3f47-4bda-bd82-89737d7545ae None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:24.104213 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab9756-fca3-4891-9412-81f9a22b860a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:24.104213 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab9756-fca3-4891-9412-81f9a22b860a None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:24.104213 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab9756-fca3-4891-9412-81f9a22b860a None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:24.104213 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab9756-fca3-4891-9412-81f9a22b860a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:24.104617 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab9756-fca3-4891-9412-81f9a22b860a None admin] RBAC: Adding query filter params (role.id=81d82471e2be4f0aa8164b75c16a9718, scope.project.id=839585501f414e1ab02501422510e54f, user.id=e12ef3e115544a51a6bf9b08d265fefb) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:24.104617 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab9756-fca3-4891-9412-81f9a22b860a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:24.104617 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab9756-fca3-4891-9412-81f9a22b860a None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:24.105007 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 264/535] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:19:23 2022] GET /identity/v3/role_assignments?user.id=b83535b443a84275a160cd84cb3ad5a0&scope.project.id=839585501f414e1ab02501422510e54f&role.id=81d82471e2be4f0aa8164b75c16a9718 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:24.106521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadf1bc4-3f47-4bda-bd82-89737d7545ae None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:24.110845 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 272/536] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:19:23 2022] GET /identity/v3/role_assignments?user.id=e12ef3e115544a51a6bf9b08d265fefb&scope.project.id=839585501f414e1ab02501422510e54f&role.id=81d82471e2be4f0aa8164b75c16a9718 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:24.264660 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadf1bc4-3f47-4bda-bd82-89737d7545ae None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:24.266399 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:24.320233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c659570a-69f1-42a0-863c-5cafd13089eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:24.321229 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c659570a-69f1-42a0-863c-5cafd13089eb None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:24.321589 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c659570a-69f1-42a0-863c-5cafd13089eb None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:24.321942 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c659570a-69f1-42a0-863c-5cafd13089eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:24.334582 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c659570a-69f1-42a0-863c-5cafd13089eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:24.336202 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c659570a-69f1-42a0-863c-5cafd13089eb None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:24.355243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-c659570a-69f1-42a0-863c-5cafd13089eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:24.355243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:24.355243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:24.355243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:24.355243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:24.355243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:24.355243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:24.355243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:24.355243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:24.355243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:24.355243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:24.355243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:24.355243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:24.355243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:24.355243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:24.356158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:24.356158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:24.356158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:24.356158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:24.356158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:24.356158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:24.356158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:24.356158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:24.356158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:24.356158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:24.356158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:24.356158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:24.356158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:24.356158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:24.356158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:24.356158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:24.356158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:24.357369 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:24.357369 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:24.357369 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:24.357369 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:24.357369 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 265/537] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:24.362400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab9756-fca3-4891-9412-81f9a22b860a None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:24.477811 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab9756-fca3-4891-9412-81f9a22b860a None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:24.524928 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:24.536667 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320eb65-ce31-43d2-9042-9d242129841d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:24.537956 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320eb65-ce31-43d2-9042-9d242129841d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:24.538335 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320eb65-ce31-43d2-9042-9d242129841d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:24.538626 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320eb65-ce31-43d2-9042-9d242129841d None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:24.539458 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320eb65-ce31-43d2-9042-9d242129841d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:24.539776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320eb65-ce31-43d2-9042-9d242129841d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:24.541243 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320eb65-ce31-43d2-9042-9d242129841d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:24.555972 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 273/538] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:24.561294 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c659570a-69f1-42a0-863c-5cafd13089eb None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:24.695464 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c659570a-69f1-42a0-863c-5cafd13089eb None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:24.735687 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a300b0-0570-4fb7-bd43-7d015ac776e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:24.736520 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a300b0-0570-4fb7-bd43-7d015ac776e0 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:24.736808 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a300b0-0570-4fb7-bd43-7d015ac776e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:24.737087 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a300b0-0570-4fb7-bd43-7d015ac776e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:24.745207 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a300b0-0570-4fb7-bd43-7d015ac776e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:24.746688 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a300b0-0570-4fb7-bd43-7d015ac776e0 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:24.766833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-11a300b0-0570-4fb7-bd43-7d015ac776e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:24.766833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:24.766833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:24.766833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:24.766833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:24.766833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:24.766833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:24.766833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:24.766833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:24.766833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:24.766833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:24.766833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:24.766833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:19:24.766833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:19:24.766833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:19:24.767751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:19:24.767751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:24.767751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:24.767751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:24.767751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:24.767751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:24.767751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:24.767751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:24.767751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:24.767751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:24.767751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:24.767751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:24.767751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:24.767751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:24.767751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:24.767751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:24.767751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:24.769519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:19:24.769519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:19:24.769519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:19:24.769519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:19:24.769519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:19:24.769519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:19:24.769519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:24.769519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:24.769519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 266/539] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:19:24.774219 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320eb65-ce31-43d2-9042-9d242129841d None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:24.833276 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d60e5e-6831-4e9d-bd5c-6691bf319045 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:24.833460 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d60e5e-6831-4e9d-bd5c-6691bf319045 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:24.834330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d60e5e-6831-4e9d-bd5c-6691bf319045 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:24.836341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 267/540] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:24.841735 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fc254-bd9d-4458-9e7f-26c4d0560d92 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:24.842032 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fc254-bd9d-4458-9e7f-26c4d0560d92 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:24.842315 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8fc254-bd9d-4458-9e7f-26c4d0560d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:24.862074 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-0d8fc254-bd9d-4458-9e7f-26c4d0560d92 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:24.863272 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-0d8fc254-bd9d-4458-9e7f-26c4d0560d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:24.874278 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:24.955650 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 268/541] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:24.992218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320eb65-ce31-43d2-9042-9d242129841d None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:25.042380 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fc254-bd9d-4458-9e7f-26c4d0560d92 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:25.050294 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a015c-1193-4c46-976d-a5fe9fdea8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:25.052981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a015c-1193-4c46-976d-a5fe9fdea8b3 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:25.053474 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a015c-1193-4c46-976d-a5fe9fdea8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:25.053856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a015c-1193-4c46-976d-a5fe9fdea8b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:25.054661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a015c-1193-4c46-976d-a5fe9fdea8b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:25.054973 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a015c-1193-4c46-976d-a5fe9fdea8b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:25.056445 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a015c-1193-4c46-976d-a5fe9fdea8b3 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:25.089793 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 274/542] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:19:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:25.094929 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a015c-1193-4c46-976d-a5fe9fdea8b3 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:25.187278 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:25.232841 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a015c-1193-4c46-976d-a5fe9fdea8b3 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:25.268963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950a2ec-8d55-412c-82fa-6870c3801f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:25.269624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950a2ec-8d55-412c-82fa-6870c3801f75 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:25.269907 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950a2ec-8d55-412c-82fa-6870c3801f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:25.270187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950a2ec-8d55-412c-82fa-6870c3801f75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:25.271011 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7950a2ec-8d55-412c-82fa-6870c3801f75 None admin] RBAC: Adding query filter params (role.id=81d82471e2be4f0aa8164b75c16a9718, scope.project.id=839585501f414e1ab02501422510e54f, user.id=b6677be3ddf647e2b971bb3ba2c34a3a) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:25.272141 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7950a2ec-8d55-412c-82fa-6870c3801f75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:25.273936 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7950a2ec-8d55-412c-82fa-6870c3801f75 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:25.280125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 275/543] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:19:25 2022] GET /identity/v3/role_assignments?user.id=b6677be3ddf647e2b971bb3ba2c34a3a&scope.project.id=839585501f414e1ab02501422510e54f&role.id=81d82471e2be4f0aa8164b75c16a9718 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:25.303543 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8fc254-bd9d-4458-9e7f-26c4d0560d92 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:25.375908 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5c2317-8916-42d7-9585-1c64dce9c7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:25.376767 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c2317-8916-42d7-9585-1c64dce9c7ca None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:25.377059 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c2317-8916-42d7-9585-1c64dce9c7ca None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:25.377342 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5c2317-8916-42d7-9585-1c64dce9c7ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:25.397258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5c2317-8916-42d7-9585-1c64dce9c7ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:25.405251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-1f5c2317-8916-42d7-9585-1c64dce9c7ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:25.405251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:25.405251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:25.405251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:25.405251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:25.405251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:25.405251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:25.405251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:25.405251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:25.405251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:25.405251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:25.405251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:25.405251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:25.405251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:25.405251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:25.405251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:25.406209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:25.406209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:25.406209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:25.406209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:25.406209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:25.406209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:25.406209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:25.406209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:25.406209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:25.406209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:25.406209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:25.406209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:25.406209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:25.406209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:25.406209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:25.406209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:25.407610 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:25.407610 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:25.407610 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:25.407610 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:25.407610 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:25.407610 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:25.407610 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:25.407610 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:25.407610 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:25.407610 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:25.407610 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:25.407610 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:25.407610 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 269/544] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 12:19:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:19:25.413377 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950a2ec-8d55-412c-82fa-6870c3801f75 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:25.543006 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76920e3-b986-4b65-8b90-bbc9b1c96be4 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:25.543350 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76920e3-b986-4b65-8b90-bbc9b1c96be4 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:25.543578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76920e3-b986-4b65-8b90-bbc9b1c96be4 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:25.545253 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 270/545] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:25.550192 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923cd14f-4897-4004-88dd-1fd18c844ec9 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:25.550590 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923cd14f-4897-4004-88dd-1fd18c844ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:25.551127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923cd14f-4897-4004-88dd-1fd18c844ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:25.565575 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:25.592445 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-923cd14f-4897-4004-88dd-1fd18c844ec9 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:25.593737 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-923cd14f-4897-4004-88dd-1fd18c844ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:25.594759 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950a2ec-8d55-412c-82fa-6870c3801f75 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:25.611408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:25.640737 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145793ae-3cd6-4d62-a5c3-1a55950eebfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:25.641337 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145793ae-3cd6-4d62-a5c3-1a55950eebfa None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:25.641626 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145793ae-3cd6-4d62-a5c3-1a55950eebfa None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:25.641910 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145793ae-3cd6-4d62-a5c3-1a55950eebfa None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:25.642675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145793ae-3cd6-4d62-a5c3-1a55950eebfa None admin] RBAC: Adding query filter params (name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:25.642989 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145793ae-3cd6-4d62-a5c3-1a55950eebfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:25.644414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145793ae-3cd6-4d62-a5c3-1a55950eebfa None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:25.656151 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 276/546] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 12:19:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:25.661463 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145793ae-3cd6-4d62-a5c3-1a55950eebfa None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:25.770801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 271/547] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 12:19:25.781667 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145793ae-3cd6-4d62-a5c3-1a55950eebfa None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:25.811708 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923cd14f-4897-4004-88dd-1fd18c844ec9 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:25.823667 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e24179-ff4a-4bdf-8657-07dacafc0b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:25.824628 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e24179-ff4a-4bdf-8657-07dacafc0b3c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:25.825017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e24179-ff4a-4bdf-8657-07dacafc0b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:25.825421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e24179-ff4a-4bdf-8657-07dacafc0b3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:25.848532 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e24179-ff4a-4bdf-8657-07dacafc0b3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:25.850870 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e24179-ff4a-4bdf-8657-07dacafc0b3c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:25.881745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-85e24179-ff4a-4bdf-8657-07dacafc0b3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:25.881745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:25.881745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:25.881745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:25.881745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:25.881745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:25.881745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:25.881745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:25.881745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:25.881745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:25.881745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:25.881745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:25.881745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:25.881745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:25.881745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:25.882802 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:25.882802 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:25.882802 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:25.882802 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:25.882802 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:25.882802 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:25.882802 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:25.882802 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:25.882802 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:25.882802 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:25.882802 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:25.882802 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:25.882802 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:25.882802 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:25.882802 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:25.882802 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:25.882802 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:25.883939 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:25.883939 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:25.883939 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:25.883939 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:25.885465 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 277/548] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:25.890821 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e24179-ff4a-4bdf-8657-07dacafc0b3c None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:25.919713 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923cd14f-4897-4004-88dd-1fd18c844ec9 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:25.938326 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:25.956141 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba633e-9575-4dea-8193-b794b5e7a6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:25.957016 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba633e-9575-4dea-8193-b794b5e7a6ff None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:25.957303 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba633e-9575-4dea-8193-b794b5e7a6ff None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:25.957577 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba633e-9575-4dea-8193-b794b5e7a6ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:25.960478 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:25.971246 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcba633e-9575-4dea-8193-b794b5e7a6ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:25.980063 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-fcba633e-9575-4dea-8193-b794b5e7a6ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:25.980063 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:25.980063 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:25.980063 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:25.980063 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:25.980063 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:25.980063 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:25.980063 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:25.980063 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:25.980063 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:25.980063 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:25.980063 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:25.980063 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:25.980063 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:25.980063 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:25.980063 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:25.982495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:25.982495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:25.982495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:25.982495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:25.982495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:25.982495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:25.982495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:25.982495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:25.982495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:25.982495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:25.982495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:25.982495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:25.982495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:25.982495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:25.982495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:25.982495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:25.984524 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:25.984524 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:25.984524 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:25.984524 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:25.984524 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:25.984524 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:25.984524 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:25.984524 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:25.984524 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:25.984524 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:25.984524 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:25.984524 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:25.984524 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 272/549] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 12:19:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:25.991895 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba633e-9575-4dea-8193-b794b5e7a6ff None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:26.094933 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e24179-ff4a-4bdf-8657-07dacafc0b3c None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:26.106748 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba633e-9575-4dea-8193-b794b5e7a6ff None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:26.144289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47229e16-e84e-498a-a3ea-a3998bb7bf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:26.144892 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47229e16-e84e-498a-a3ea-a3998bb7bf94 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:26.145184 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47229e16-e84e-498a-a3ea-a3998bb7bf94 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:26.145466 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47229e16-e84e-498a-a3ea-a3998bb7bf94 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:26.146260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47229e16-e84e-498a-a3ea-a3998bb7bf94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:26.146575 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47229e16-e84e-498a-a3ea-a3998bb7bf94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:26.147956 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47229e16-e84e-498a-a3ea-a3998bb7bf94 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:26.169824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 278/550] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:26.175723 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47229e16-e84e-498a-a3ea-a3998bb7bf94 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:26.178730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a21fb5-87ad-4207-8dc2-96f58594f26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:26.179677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a21fb5-87ad-4207-8dc2-96f58594f26a None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:26.180212 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a21fb5-87ad-4207-8dc2-96f58594f26a None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:26.180721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a21fb5-87ad-4207-8dc2-96f58594f26a None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:26.181680 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a21fb5-87ad-4207-8dc2-96f58594f26a None admin] RBAC: Adding query filter params (name=service) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:26.182190 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a21fb5-87ad-4207-8dc2-96f58594f26a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:26.183786 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a21fb5-87ad-4207-8dc2-96f58594f26a None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:26.193757 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 273/551] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 12:19:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:19:26.199434 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a21fb5-87ad-4207-8dc2-96f58594f26a None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:26.304022 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:26.310776 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:26.326726 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47229e16-e84e-498a-a3ea-a3998bb7bf94 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:26.390924 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813d5cb-525c-494b-ad97-e623739cae6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:26.391980 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813d5cb-525c-494b-ad97-e623739cae6e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:26.392517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813d5cb-525c-494b-ad97-e623739cae6e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:26.392943 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813d5cb-525c-494b-ad97-e623739cae6e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:26.411002 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a21fb5-87ad-4207-8dc2-96f58594f26a None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:26.412532 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0813d5cb-525c-494b-ad97-e623739cae6e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:26.415875 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0813d5cb-525c-494b-ad97-e623739cae6e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:26.453137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-0813d5cb-525c-494b-ad97-e623739cae6e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 12:19:26.453137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:26.453137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:26.453137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:26.453137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:26.453137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:26.453137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:26.453137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:26.453137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:26.453137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:26.453137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:26.453137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:26.453137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:19:26.453137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:19:26.453137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:19:26.453137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:19:26.454414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:26.454414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:26.454414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:19:26.454414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:26.454414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:19:26.454414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:26.454414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:26.454414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:26.454414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:26.454414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:26.454414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:26.454414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:26.454414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:26.454414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:26.454414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:26.454414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:26.454414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:26.454414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:26.455728 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:26.455728 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:26.455728 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:19:26.455728 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:19:26.455728 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:19:26.455728 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:19:26.455728 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:19:26.455728 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:19:26.455728 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 12:19:26.455728 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:26.455728 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 279/552] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:19:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:26.464194 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813d5cb-525c-494b-ad97-e623739cae6e None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:26.472905 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbadd96-1b5d-46cf-bc4c-b6ba086f4c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:26.473905 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbadd96-1b5d-46cf-bc4c-b6ba086f4c0c None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:26.475233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbadd96-1b5d-46cf-bc4c-b6ba086f4c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:26.475694 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbadd96-1b5d-46cf-bc4c-b6ba086f4c0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:26.498289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbadd96-1b5d-46cf-bc4c-b6ba086f4c0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:26.500000 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbadd96-1b5d-46cf-bc4c-b6ba086f4c0c None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:26.519485 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-5dbadd96-1b5d-46cf-bc4c-b6ba086f4c0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:26.519485 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:26.519485 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:26.519485 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:26.519485 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:26.519485 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:26.519485 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:26.519485 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:26.519485 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:26.519485 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:26.519485 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:26.519485 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:26.519485 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:26.519485 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:26.519485 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:26.520635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:26.520635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:26.520635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:26.520635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:26.520635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:26.520635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:26.520635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:26.520635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:26.520635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:26.520635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:26.520635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:26.520635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:26.520635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:26.520635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:26.520635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:26.520635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:26.520635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:26.522436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:26.522436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:26.522436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:26.522436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:26.524535 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 274/553] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:26.533372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbadd96-1b5d-46cf-bc4c-b6ba086f4c0c None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:26.670024 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813d5cb-525c-494b-ad97-e623739cae6e None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:26.739470 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e74d4-98d1-4038-9022-beedaed98ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:26.740927 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e74d4-98d1-4038-9022-beedaed98ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:26.741418 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e74d4-98d1-4038-9022-beedaed98ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:26.741819 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e74d4-98d1-4038-9022-beedaed98ec7 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:26.742689 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e74d4-98d1-4038-9022-beedaed98ec7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:26.743002 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e74d4-98d1-4038-9022-beedaed98ec7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:26.744520 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e74d4-98d1-4038-9022-beedaed98ec7 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:26.758321 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbadd96-1b5d-46cf-bc4c-b6ba086f4c0c None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:26.771849 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:26.781745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 280/554] 10.222.0.16 () {56 vars in 1135 bytes} [Mon May 23 12:19:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:19:26.787857 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e74d4-98d1-4038-9022-beedaed98ec7 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:26.797208 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:26.828467 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3028b83a-a823-4fc7-88ff-a37d0a6c73c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:26.829153 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028b83a-a823-4fc7-88ff-a37d0a6c73c2 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:26.829448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028b83a-a823-4fc7-88ff-a37d0a6c73c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:26.829733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3028b83a-a823-4fc7-88ff-a37d0a6c73c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:26.830475 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3028b83a-a823-4fc7-88ff-a37d0a6c73c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:26.830790 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3028b83a-a823-4fc7-88ff-a37d0a6c73c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:26.832198 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3028b83a-a823-4fc7-88ff-a37d0a6c73c2 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:26.845263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 275/555] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:26.846559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f78154-ed07-4d27-93b1-40e4e12f4def None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:26.846862 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f78154-ed07-4d27-93b1-40e4e12f4def None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:26.847131 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f78154-ed07-4d27-93b1-40e4e12f4def None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:26.848722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 276/556] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:26.854129 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f78154-ed07-4d27-93b1-40e4e12f4def None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:26.901895 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e74d4-98d1-4038-9022-beedaed98ec7 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:26.953195 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3cce81-8082-44ba-8452-52c6c2e9ba3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:26.954448 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3cce81-8082-44ba-8452-52c6c2e9ba3e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:26.954934 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3cce81-8082-44ba-8452-52c6c2e9ba3e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:26.955890 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3cce81-8082-44ba-8452-52c6c2e9ba3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:26.976737 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f78154-ed07-4d27-93b1-40e4e12f4def None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:27.014152 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aa2624-e7db-42c0-b89d-57aa6e3d7dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:27.015371 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa2624-e7db-42c0-b89d-57aa6e3d7dad None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:27.015731 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa2624-e7db-42c0-b89d-57aa6e3d7dad None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:27.015995 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aa2624-e7db-42c0-b89d-57aa6e3d7dad None admin] PATH_INFO: `/v3/users/glance` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:27.026644 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aa2624-e7db-42c0-b89d-57aa6e3d7dad None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:27.027965 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aa2624-e7db-42c0-b89d-57aa6e3d7dad None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:27.037588 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3cce81-8082-44ba-8452-52c6c2e9ba3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:27.039108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3cce81-8082-44ba-8452-52c6c2e9ba3e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:27.043771 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-36aa2624-e7db-42c0-b89d-57aa6e3d7dad None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 23 12:19:27.043771 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:27.043771 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:27.043771 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:27.043771 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:27.043771 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:27.043771 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:27.043771 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:27.043771 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:27.043771 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:27.043771 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:27.043771 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:27.043771 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:19:27.043771 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:19:27.043771 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:19:27.043771 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:19:27.045327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:27.045327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:27.045327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:19:27.045327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:27.045327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:19:27.045327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:27.045327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:27.045327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:27.045327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:27.045327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:27.045327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:27.045327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:27.045327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:27.045327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:27.045327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:27.045327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:27.045327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:27.045327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:27.046274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:27.046274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:27.046274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:19:27.046274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:19:27.046274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:19:27.046274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:19:27.046274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:19:27.046274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:19:27.046274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 23 12:19:27.046274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:27.048022 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:27.049575 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 277/557] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:19:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:19:27.050857 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab628a-08bf-42c1-9bbb-ff71f854e96a None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:27.051138 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab628a-08bf-42c1-9bbb-ff71f854e96a None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:27.051440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab628a-08bf-42c1-9bbb-ff71f854e96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:27.069359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-eeab628a-08bf-42c1-9bbb-ff71f854e96a None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:27.070539 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-eeab628a-08bf-42c1-9bbb-ff71f854e96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:27.093976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:27.149686 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 281/558] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:19:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:19:27.149686 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b450a-96be-45ca-b9b1-d459480e3de5 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:27.149686 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b450a-96be-45ca-b9b1-d459480e3de5 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:27.149686 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b450a-96be-45ca-b9b1-d459480e3de5 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:27.149686 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 282/559] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:27.149686 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b450a-96be-45ca-b9b1-d459480e3de5 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:27.157158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 278/560] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:27.157874 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab628a-08bf-42c1-9bbb-ff71f854e96a None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:27.265933 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b450a-96be-45ca-b9b1-d459480e3de5 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:27.268241 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab628a-08bf-42c1-9bbb-ff71f854e96a None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:27.274765 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:27.304177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6418419-2586-4fa3-a8b1-956ab8f972da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:27.304873 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6418419-2586-4fa3-a8b1-956ab8f972da None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:27.305161 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6418419-2586-4fa3-a8b1-956ab8f972da None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:27.305442 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6418419-2586-4fa3-a8b1-956ab8f972da None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:27.305759 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cba360-c948-4523-be92-617f8e290b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:27.306282 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6418419-2586-4fa3-a8b1-956ab8f972da None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:27.306555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cba360-c948-4523-be92-617f8e290b34 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:27.306650 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6418419-2586-4fa3-a8b1-956ab8f972da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:27.306840 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cba360-c948-4523-be92-617f8e290b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:27.307119 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cba360-c948-4523-be92-617f8e290b34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:27.308004 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6418419-2586-4fa3-a8b1-956ab8f972da None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:27.318662 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cba360-c948-4523-be92-617f8e290b34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:27.320341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cba360-c948-4523-be92-617f8e290b34 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:27.331875 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 283/561] 10.222.0.16 () {56 vars in 1137 bytes} [Mon May 23 12:19:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:27.331875 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30667ae-0bf7-4810-8c8e-d6b3af531fb8 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:27.331875 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30667ae-0bf7-4810-8c8e-d6b3af531fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:27.331875 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30667ae-0bf7-4810-8c8e-d6b3af531fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:27.338075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-b1cba360-c948-4523-be92-617f8e290b34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:27.338075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:27.338075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:27.338075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:27.338075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:27.338075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:27.338075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:27.338075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:27.338075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:27.338075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:27.338075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:27.338075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:27.338075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:27.338075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:27.338075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:27.338953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:27.338953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:27.338953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:27.338953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:27.338953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:27.338953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:27.338953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:27.338953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:27.338953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:27.338953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:27.338953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:27.338953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:27.338953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:27.338953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:27.338953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:27.338953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:27.338953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:27.339870 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:27.339870 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:27.339870 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:27.339870 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:27.339870 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 279/562] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:27.340598 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cba360-c948-4523-be92-617f8e290b34 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:27.352474 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-e30667ae-0bf7-4810-8c8e-d6b3af531fb8 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:27.353680 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e30667ae-0bf7-4810-8c8e-d6b3af531fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:27.355517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:27.426072 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 284/563] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:27.427019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30667ae-0bf7-4810-8c8e-d6b3af531fb8 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:27.466314 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cba360-c948-4523-be92-617f8e290b34 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:27.501965 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a479601-7bc2-43fc-bfdc-d20c238cab89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:27.502765 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a479601-7bc2-43fc-bfdc-d20c238cab89 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:27.503221 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a479601-7bc2-43fc-bfdc-d20c238cab89 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:27.503420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a479601-7bc2-43fc-bfdc-d20c238cab89 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:27.514082 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a479601-7bc2-43fc-bfdc-d20c238cab89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:27.522730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-8a479601-7bc2-43fc-bfdc-d20c238cab89 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:27.522730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:27.522730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:27.522730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:27.522730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:27.522730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:27.522730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:27.522730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:27.522730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:27.522730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:27.522730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:27.522730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:27.522730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:27.522730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:27.522730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:27.522730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:27.523671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:27.523671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:27.523671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:27.523671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:27.523671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:27.523671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:27.523671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:27.523671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:27.523671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:27.523671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:27.523671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:27.523671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:27.523671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:27.523671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:27.523671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:27.523671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:27.524555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:27.524555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:27.524555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:27.524555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:27.524555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:27.524555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:27.524555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:27.524555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:27.524555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:27.524555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:27.524555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:27.524555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:27.524555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 280/564] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 12:19:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:27.525245 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a479601-7bc2-43fc-bfdc-d20c238cab89 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:27.545460 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:27.555664 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30667ae-0bf7-4810-8c8e-d6b3af531fb8 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:27.597260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104338ea-cd66-42ee-803b-3670a7a4b9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:27.598054 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104338ea-cd66-42ee-803b-3670a7a4b9bb None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:27.598343 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104338ea-cd66-42ee-803b-3670a7a4b9bb None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:27.598685 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104338ea-cd66-42ee-803b-3670a7a4b9bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:27.603489 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:27.609241 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104338ea-cd66-42ee-803b-3670a7a4b9bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:27.611135 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104338ea-cd66-42ee-803b-3670a7a4b9bb None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:27.631455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-104338ea-cd66-42ee-803b-3670a7a4b9bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:27.631455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:27.631455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:27.631455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:27.631455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:27.631455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:27.631455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:27.631455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:27.631455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:27.631455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:27.631455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:27.631455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:27.631455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:27.631455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:27.631455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:27.637533 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:27.637533 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:27.637533 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:27.637533 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:27.637533 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:27.637533 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:27.637533 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:27.637533 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:27.637533 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:27.637533 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:27.637533 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:27.637533 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:27.637533 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:27.637533 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:27.637533 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:27.637533 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:27.637533 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:27.638447 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:27.638447 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:27.638447 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:27.638447 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:27.638447 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 285/565] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:27.638447 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104338ea-cd66-42ee-803b-3670a7a4b9bb None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:27.644390 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a479601-7bc2-43fc-bfdc-d20c238cab89 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:27.681638 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64b299-8530-44e1-9df7-68537ea4617f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:27.682311 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64b299-8530-44e1-9df7-68537ea4617f None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:27.682596 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64b299-8530-44e1-9df7-68537ea4617f None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:27.682877 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64b299-8530-44e1-9df7-68537ea4617f None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:27.683600 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64b299-8530-44e1-9df7-68537ea4617f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:27.683904 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64b299-8530-44e1-9df7-68537ea4617f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:27.685319 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64b299-8530-44e1-9df7-68537ea4617f None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:27.698404 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 281/566] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:27.699203 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64b299-8530-44e1-9df7-68537ea4617f None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:27.746665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104338ea-cd66-42ee-803b-3670a7a4b9bb None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:27.782033 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c4b20-aa39-4d45-add3-2a2e12ee7b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:27.782752 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c4b20-aa39-4d45-add3-2a2e12ee7b61 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:27.783038 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c4b20-aa39-4d45-add3-2a2e12ee7b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:27.783316 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c4b20-aa39-4d45-add3-2a2e12ee7b61 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:27.793591 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c4b20-aa39-4d45-add3-2a2e12ee7b61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:27.801338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-460c4b20-aa39-4d45-add3-2a2e12ee7b61 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:27.801338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:27.801338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:27.801338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:27.801338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:27.801338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:27.801338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:27.801338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:27.801338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:27.801338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:27.801338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:27.801338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:27.801338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:27.801338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:27.801338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:27.801338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:27.802230 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:27.802230 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:27.802230 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:27.802230 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:27.802230 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:27.802230 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:27.802230 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:27.802230 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:27.802230 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:27.802230 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:27.802230 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:27.802230 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:27.802230 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:27.802230 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:27.802230 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:27.802230 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:27.803341 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:27.803341 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:27.803341 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:27.803341 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:27.803341 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:27.803341 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:27.803341 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:27.803341 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:27.803341 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:27.803341 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:27.803341 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:27.803341 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:27.803341 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 286/567] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 12:19:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:27.804939 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c4b20-aa39-4d45-add3-2a2e12ee7b61 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:27.813859 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:27.814334 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64b299-8530-44e1-9df7-68537ea4617f None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:27.844742 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:27.851252 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26d24f-20b3-4d90-b93b-150768b3116b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:27.851920 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26d24f-20b3-4d90-b93b-150768b3116b None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:27.852243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26d24f-20b3-4d90-b93b-150768b3116b None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:27.852533 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26d24f-20b3-4d90-b93b-150768b3116b None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:27.853281 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26d24f-20b3-4d90-b93b-150768b3116b None admin] RBAC: Adding query filter params (name=service) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:27.853594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26d24f-20b3-4d90-b93b-150768b3116b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:27.854967 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26d24f-20b3-4d90-b93b-150768b3116b None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:27.862764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 282/568] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 12:19:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:27.863702 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26d24f-20b3-4d90-b93b-150768b3116b None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:27.905670 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c4b20-aa39-4d45-add3-2a2e12ee7b61 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:27.939402 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d5f83-b47d-46fe-8423-b09aa462c0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:27.940053 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d5f83-b47d-46fe-8423-b09aa462c0a5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:27.940365 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d5f83-b47d-46fe-8423-b09aa462c0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:27.940643 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d5f83-b47d-46fe-8423-b09aa462c0a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:27.941358 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5d5f83-b47d-46fe-8423-b09aa462c0a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:27.941660 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5d5f83-b47d-46fe-8423-b09aa462c0a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:27.944007 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5d5f83-b47d-46fe-8423-b09aa462c0a5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:27.955684 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 287/569] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:27.956739 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9258d0db-ff61-4b75-a497-9d5951288fe6 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:27.957033 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9258d0db-ff61-4b75-a497-9d5951288fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:27.957310 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9258d0db-ff61-4b75-a497-9d5951288fe6 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:27.960226 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 288/570] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:27.960226 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9258d0db-ff61-4b75-a497-9d5951288fe6 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:27.968903 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26d24f-20b3-4d90-b93b-150768b3116b None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:28.002234 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b874f-eacc-4d2f-9159-5b571844b5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:28.003025 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b874f-eacc-4d2f-9159-5b571844b5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:28.003309 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b874f-eacc-4d2f-9159-5b571844b5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:28.003588 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b874f-eacc-4d2f-9159-5b571844b5e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:28.011139 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b874f-eacc-4d2f-9159-5b571844b5e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:28.012623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b874f-eacc-4d2f-9159-5b571844b5e4 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:28.027219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-1d2b874f-eacc-4d2f-9159-5b571844b5e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:28.027219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:28.027219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:28.027219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:28.027219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:28.027219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:28.027219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:28.027219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:28.027219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:28.027219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:28.027219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:28.027219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:28.027219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:19:28.027219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:19:28.027219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:19:28.028291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:19:28.028291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:28.028291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:28.028291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:28.028291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:28.028291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:28.028291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:28.028291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:28.028291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:28.028291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:28.028291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:28.028291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:28.028291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:28.028291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:28.028291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:28.028291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:28.028291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:28.029249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:19:28.029249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:19:28.029249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:19:28.029249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:19:28.029249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:19:28.029249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:19:28.029249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:28.029249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:28.029249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 283/571] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:28.029735 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b874f-eacc-4d2f-9159-5b571844b5e4 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:28.030971 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:28.067498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9258d0db-ff61-4b75-a497-9d5951288fe6 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:28.087035 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:28.101176 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cadf4-7b6a-4b96-8192-5b9adb6a7b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:28.101869 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cadf4-7b6a-4b96-8192-5b9adb6a7b3c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:28.102156 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cadf4-7b6a-4b96-8192-5b9adb6a7b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:28.102436 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cadf4-7b6a-4b96-8192-5b9adb6a7b3c None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:28.103178 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8cadf4-7b6a-4b96-8192-5b9adb6a7b3c None admin] RBAC: Adding query filter params (name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:28.103513 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8cadf4-7b6a-4b96-8192-5b9adb6a7b3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:28.104909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8cadf4-7b6a-4b96-8192-5b9adb6a7b3c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:28.112737 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 289/572] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 12:19:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:28.113516 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cadf4-7b6a-4b96-8192-5b9adb6a7b3c None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:28.137280 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b874f-eacc-4d2f-9159-5b571844b5e4 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:28.170847 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263f68ec-3739-47da-a4be-1ffb73bb2bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:28.171632 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f68ec-3739-47da-a4be-1ffb73bb2bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:28.171924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f68ec-3739-47da-a4be-1ffb73bb2bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:28.172230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f68ec-3739-47da-a4be-1ffb73bb2bb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:28.182867 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263f68ec-3739-47da-a4be-1ffb73bb2bb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:28.184356 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263f68ec-3739-47da-a4be-1ffb73bb2bb5 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:28.202970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-263f68ec-3739-47da-a4be-1ffb73bb2bb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:28.202970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:28.202970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:28.202970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:28.202970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:28.202970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:28.202970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:28.202970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:28.202970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:28.202970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:28.202970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:28.202970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:28.202970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:28.202970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:28.202970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:28.203968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:28.203968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:28.203968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:28.203968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:28.203968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:28.203968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:28.203968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:28.203968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:28.203968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:28.203968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:28.203968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:28.203968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:28.203968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:28.203968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:28.203968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:28.203968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:28.203968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:28.210000 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:28.210000 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:28.210000 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:28.210000 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:28.210000 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 284/573] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:28.210000 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b18f0b-1df0-44e1-a9e2-4989d804911c None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:28.210000 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b18f0b-1df0-44e1-a9e2-4989d804911c None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:28.210000 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b18f0b-1df0-44e1-a9e2-4989d804911c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:28.223150 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cadf4-7b6a-4b96-8192-5b9adb6a7b3c None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:28.228532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-79b18f0b-1df0-44e1-a9e2-4989d804911c None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:28.229743 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-79b18f0b-1df0-44e1-a9e2-4989d804911c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:28.250350 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:28.259805 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6207046-2d9e-4557-af45-cd53ae98597e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:28.260644 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6207046-2d9e-4557-af45-cd53ae98597e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:28.261174 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6207046-2d9e-4557-af45-cd53ae98597e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:28.261313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6207046-2d9e-4557-af45-cd53ae98597e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:28.268037 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6207046-2d9e-4557-af45-cd53ae98597e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:28.269525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6207046-2d9e-4557-af45-cd53ae98597e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:28.284406 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-e6207046-2d9e-4557-af45-cd53ae98597e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:28.284406 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:28.284406 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:28.284406 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:28.284406 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:28.284406 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:28.284406 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:28.284406 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:28.284406 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:28.284406 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:28.284406 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:28.284406 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:28.284406 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:19:28.284406 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:19:28.284406 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:19:28.285454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:19:28.285454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:28.285454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:28.285454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:28.285454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:28.285454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:28.285454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:28.285454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:28.285454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:28.285454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:28.285454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:28.285454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:28.285454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:28.285454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:28.285454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:28.285454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:28.285454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:28.286550 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:19:28.286550 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:19:28.286550 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:19:28.286550 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:19:28.286550 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:19:28.286550 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:19:28.286550 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:28.286550 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:28.286550 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 290/574] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:28.287318 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6207046-2d9e-4557-af45-cd53ae98597e None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:28.302243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 285/575] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:28.303590 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b18f0b-1df0-44e1-a9e2-4989d804911c None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:28.337166 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:28.390662 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6207046-2d9e-4557-af45-cd53ae98597e None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:28.402548 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b18f0b-1df0-44e1-a9e2-4989d804911c None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:28.426317 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834152d9-02cd-41a3-9591-9c24d8ef03ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:28.426976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834152d9-02cd-41a3-9591-9c24d8ef03ab None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:28.427875 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834152d9-02cd-41a3-9591-9c24d8ef03ab None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:28.428199 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834152d9-02cd-41a3-9591-9c24d8ef03ab None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:28.428986 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834152d9-02cd-41a3-9591-9c24d8ef03ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:28.429301 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834152d9-02cd-41a3-9591-9c24d8ef03ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:28.430728 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834152d9-02cd-41a3-9591-9c24d8ef03ab None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:28.437588 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9650404-9a55-437f-8915-930a41ecb9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:28.438382 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9650404-9a55-437f-8915-930a41ecb9ad None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:28.438666 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9650404-9a55-437f-8915-930a41ecb9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:28.438940 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9650404-9a55-437f-8915-930a41ecb9ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:28.442401 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 291/576] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:19:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:28.443626 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834152d9-02cd-41a3-9591-9c24d8ef03ab None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:28.450972 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9650404-9a55-437f-8915-930a41ecb9ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:28.453484 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9650404-9a55-437f-8915-930a41ecb9ad None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:28.471928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-b9650404-9a55-437f-8915-930a41ecb9ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:28.471928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:28.471928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:28.471928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:28.471928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:28.471928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:28.471928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:28.471928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:28.471928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:28.471928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:28.471928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:28.471928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:28.471928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:28.471928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:28.471928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:28.472924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:28.472924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:28.472924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:28.472924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:28.472924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:28.472924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:28.472924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:28.472924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:28.472924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:28.472924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:28.472924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:28.472924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:28.472924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:28.472924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:28.472924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:28.472924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:28.472924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:28.473831 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:28.473831 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:28.473831 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:28.473831 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:28.473831 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 286/577] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:28.474659 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9650404-9a55-437f-8915-930a41ecb9ad None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:28.495710 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:28.551482 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834152d9-02cd-41a3-9591-9c24d8ef03ab None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:28.561597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:28.577968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9650404-9a55-437f-8915-930a41ecb9ad None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:28.597257 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db7b3c-88f8-4b12-8a83-c10bf2bb8449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:28.597846 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db7b3c-88f8-4b12-8a83-c10bf2bb8449 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:28.598129 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db7b3c-88f8-4b12-8a83-c10bf2bb8449 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:28.598413 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db7b3c-88f8-4b12-8a83-c10bf2bb8449 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:28.599839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82db7b3c-88f8-4b12-8a83-c10bf2bb8449 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:28.600187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82db7b3c-88f8-4b12-8a83-c10bf2bb8449 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:28.601552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82db7b3c-88f8-4b12-8a83-c10bf2bb8449 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:28.612416 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6996d-7727-4772-a5aa-4bbf51b51521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:28.613064 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 292/578] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:28.613164 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6996d-7727-4772-a5aa-4bbf51b51521 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:28.613350 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6996d-7727-4772-a5aa-4bbf51b51521 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:28.613632 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6996d-7727-4772-a5aa-4bbf51b51521 None admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:28.613854 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db7b3c-88f8-4b12-8a83-c10bf2bb8449 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:28.614418 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf6996d-7727-4772-a5aa-4bbf51b51521 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:28.614731 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf6996d-7727-4772-a5aa-4bbf51b51521 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:28.616909 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf6996d-7727-4772-a5aa-4bbf51b51521 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:28.628665 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 287/579] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:19:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:28.629457 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6996d-7727-4772-a5aa-4bbf51b51521 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:28.719840 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db7b3c-88f8-4b12-8a83-c10bf2bb8449 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:28.729459 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:28.730823 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6996d-7727-4772-a5aa-4bbf51b51521 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:28.753543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02620e19-bfa5-447b-933f-84ed20f76e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:28.754332 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02620e19-bfa5-447b-933f-84ed20f76e62 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:28.754615 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02620e19-bfa5-447b-933f-84ed20f76e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:28.754885 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02620e19-bfa5-447b-933f-84ed20f76e62 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:28.763689 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb4faa-a4d0-4567-b3cc-15645f42edb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:28.764509 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb4faa-a4d0-4567-b3cc-15645f42edb9 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:28.764794 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb4faa-a4d0-4567-b3cc-15645f42edb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:28.764890 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02620e19-bfa5-447b-933f-84ed20f76e62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:28.765071 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb4faa-a4d0-4567-b3cc-15645f42edb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:28.765901 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb4faa-a4d0-4567-b3cc-15645f42edb9 None admin] RBAC: Adding query filter params (role.id=81d82471e2be4f0aa8164b75c16a9718, scope.project.id=839585501f414e1ab02501422510e54f, user.id=71fb9a79ac494635804b93d4a0048205) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:28.766213 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb4faa-a4d0-4567-b3cc-15645f42edb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:28.767662 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb4faa-a4d0-4567-b3cc-15645f42edb9 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:28.773313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-02620e19-bfa5-447b-933f-84ed20f76e62 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:28.773313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:28.773313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:28.773313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:28.773313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:28.773313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:28.773313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:28.773313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:28.773313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:28.773313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:28.773313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:28.773313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:28.773313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:28.773313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:28.773313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:28.773313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:28.785105 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:28.785105 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:28.785105 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:28.785105 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:28.785105 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:28.785105 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:28.785105 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:28.785105 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:28.785105 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:28.785105 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:28.785105 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:28.785105 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:28.785105 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:28.785105 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:28.785105 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:28.785105 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:28.785962 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:28.785962 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:28.785962 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:28.785962 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:28.785962 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:28.785962 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:28.785962 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:28.785962 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:28.785962 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:28.785962 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:28.785962 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:28.785962 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:28.786614 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 288/580] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:19:28 2022] GET /identity/v3/role_assignments?user.id=71fb9a79ac494635804b93d4a0048205&scope.project.id=839585501f414e1ab02501422510e54f&role.id=81d82471e2be4f0aa8164b75c16a9718 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:28.787465 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 293/581] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 12:19:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:28.787550 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb4faa-a4d0-4567-b3cc-15645f42edb9 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:28.787621 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02620e19-bfa5-447b-933f-84ed20f76e62 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:28.804925 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:28.901639 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02620e19-bfa5-447b-933f-84ed20f76e62 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:28.918210 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb4faa-a4d0-4567-b3cc-15645f42edb9 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:28.938576 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5109c99-e323-4098-a2e2-a2e2da7eb355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:28.939387 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5109c99-e323-4098-a2e2-a2e2da7eb355 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:28.939671 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5109c99-e323-4098-a2e2-a2e2da7eb355 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:28.939950 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5109c99-e323-4098-a2e2-a2e2da7eb355 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:28.948972 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5109c99-e323-4098-a2e2-a2e2da7eb355 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:28.950428 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5109c99-e323-4098-a2e2-a2e2da7eb355 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:28.959270 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25bcb48-335f-4390-8b14-3ddf3ff8e3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:28.960245 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bcb48-335f-4390-8b14-3ddf3ff8e3d4 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:28.960730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bcb48-335f-4390-8b14-3ddf3ff8e3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:28.961186 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bcb48-335f-4390-8b14-3ddf3ff8e3d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:28.961917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25bcb48-335f-4390-8b14-3ddf3ff8e3d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:28.962237 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25bcb48-335f-4390-8b14-3ddf3ff8e3d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:28.963613 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25bcb48-335f-4390-8b14-3ddf3ff8e3d4 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:28.965649 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-f5109c99-e323-4098-a2e2-a2e2da7eb355 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 12:19:28.965649 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:28.965649 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:28.965649 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:28.965649 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:28.965649 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:28.965649 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:28.965649 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:28.965649 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:28.965649 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:28.965649 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:28.965649 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:28.965649 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:19:28.965649 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:19:28.965649 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:19:28.965649 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:19:28.966614 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:28.966614 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:28.966614 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:19:28.966614 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:28.966614 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:19:28.966614 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:28.966614 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:28.966614 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:28.966614 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:28.966614 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:28.966614 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:28.966614 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:28.966614 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:28.966614 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:28.966614 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:28.966614 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:28.966614 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:28.966614 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:28.967758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:28.967758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:28.967758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:19:28.967758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:19:28.967758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:19:28.967758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:19:28.967758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:19:28.967758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:19:28.967758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 12:19:28.967758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:28.967758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 294/582] 10.222.0.16 () {56 vars in 1093 bytes} [Mon May 23 12:19:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:28.968495 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5109c99-e323-4098-a2e2-a2e2da7eb355 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:28.984594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 289/583] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:28.985712 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25bcb48-335f-4390-8b14-3ddf3ff8e3d4 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:29.000004 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:29.065805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:29.087462 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5109c99-e323-4098-a2e2-a2e2da7eb355 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:29.102542 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25bcb48-335f-4390-8b14-3ddf3ff8e3d4 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:29.122725 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd39ac-f64f-40b7-8668-0c8e4fdd9b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:29.123627 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd39ac-f64f-40b7-8668-0c8e4fdd9b88 None admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:29.123914 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd39ac-f64f-40b7-8668-0c8e4fdd9b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:29.124357 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd39ac-f64f-40b7-8668-0c8e4fdd9b88 None admin] PATH_INFO: `/v3/projects/839585501f414e1ab02501422510e54f/users/2483a351f5e943ad8c872a98a4717b0b/roles/81d82471e2be4f0aa8164b75c16a9718` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:29.139865 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd39ac-f64f-40b7-8668-0c8e4fdd9b88 None admin] RBAC: Authorizing `identity:create_grant(project_id=839585501f414e1ab02501422510e54f, user_id=2483a351f5e943ad8c872a98a4717b0b, role_id=81d82471e2be4f0aa8164b75c16a9718)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:29.141569 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd39ac-f64f-40b7-8668-0c8e4fdd9b88 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:29.142090 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e08027-36f6-490a-9f35-8d168be93875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:29.142756 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e08027-36f6-490a-9f35-8d168be93875 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:29.143041 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e08027-36f6-490a-9f35-8d168be93875 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:29.143320 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e08027-36f6-490a-9f35-8d168be93875 None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:29.144265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e08027-36f6-490a-9f35-8d168be93875 None admin] RBAC: Adding query filter params (name=service) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:29.144967 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e08027-36f6-490a-9f35-8d168be93875 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:29.147584 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e08027-36f6-490a-9f35-8d168be93875 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:29.156827 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 295/584] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 12:19:28 2022] PUT /identity/v3/projects/839585501f414e1ab02501422510e54f/users/2483a351f5e943ad8c872a98a4717b0b/roles/81d82471e2be4f0aa8164b75c16a9718 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:19:29.158521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd39ac-f64f-40b7-8668-0c8e4fdd9b88 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:29.162137 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 290/585] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 12:19:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:29.163359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e08027-36f6-490a-9f35-8d168be93875 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:29.261296 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:29.290292 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd39ac-f64f-40b7-8668-0c8e4fdd9b88 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:29.303957 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e08027-36f6-490a-9f35-8d168be93875 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:29.329527 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd08270-46be-49ec-8882-2ed53a7fc193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:29.330505 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd08270-46be-49ec-8882-2ed53a7fc193 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:29.330945 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd08270-46be-49ec-8882-2ed53a7fc193 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:29.331379 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd08270-46be-49ec-8882-2ed53a7fc193 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:29.332450 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd08270-46be-49ec-8882-2ed53a7fc193 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:29.332918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd08270-46be-49ec-8882-2ed53a7fc193 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:29.334842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd08270-46be-49ec-8882-2ed53a7fc193 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:29.338742 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:29.345004 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e70c19-cfe7-493f-97c4-893aa60ed579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:29.345970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e70c19-cfe7-493f-97c4-893aa60ed579 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:29.347110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e70c19-cfe7-493f-97c4-893aa60ed579 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:29.347110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e70c19-cfe7-493f-97c4-893aa60ed579 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:29.356355 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e70c19-cfe7-493f-97c4-893aa60ed579 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:29.358107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e70c19-cfe7-493f-97c4-893aa60ed579 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:29.358309 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 296/586] 10.222.0.16 () {56 vars in 1133 bytes} [Mon May 23 12:19:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:29.359453 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd08270-46be-49ec-8882-2ed53a7fc193 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:29.375851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-48e70c19-cfe7-493f-97c4-893aa60ed579 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 12:19:29.375851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:29.375851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:29.375851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:29.375851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:29.375851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:29.375851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:29.375851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:29.375851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:29.375851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:29.375851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:29.375851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:29.375851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:19:29.375851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:19:29.375851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:19:29.375851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:19:29.376933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:29.376933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:29.376933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:19:29.376933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:29.376933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:19:29.376933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:29.376933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:29.376933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:29.376933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:29.376933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:29.376933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:29.376933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:29.376933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:29.376933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:29.376933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:29.376933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:29.376933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:29.376933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:29.377897 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:29.377897 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:29.377897 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:19:29.377897 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:19:29.377897 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:19:29.377897 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:19:29.377897 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:19:29.377897 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:19:29.377897 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 12:19:29.377897 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:29.377897 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 291/587] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:19:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:29.378500 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e70c19-cfe7-493f-97c4-893aa60ed579 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:29.476436 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd08270-46be-49ec-8882-2ed53a7fc193 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:29.513062 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e70c19-cfe7-493f-97c4-893aa60ed579 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:29.519048 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e429f-c40c-467b-8cf1-6980c9d0cd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:29.520204 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e429f-c40c-467b-8cf1-6980c9d0cd2c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:29.520674 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e429f-c40c-467b-8cf1-6980c9d0cd2c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:29.521739 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e429f-c40c-467b-8cf1-6980c9d0cd2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:29.525407 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:29.534310 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e429f-c40c-467b-8cf1-6980c9d0cd2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:29.536080 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e429f-c40c-467b-8cf1-6980c9d0cd2c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:29.555624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-a06e429f-c40c-467b-8cf1-6980c9d0cd2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:29.555624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:29.555624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:29.555624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:29.555624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:29.555624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:29.555624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:29.555624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:29.555624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:29.555624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:29.555624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:29.555624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:29.555624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:29.555624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:29.555624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:29.556622 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:29.556622 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:29.556622 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:29.556622 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:29.556622 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:29.556622 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:29.556622 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:29.556622 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:29.556622 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:29.556622 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:29.556622 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:29.556622 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:29.556622 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:29.556622 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:29.556622 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:29.556622 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:29.556622 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:29.557509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:29.557509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:29.557509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:29.557509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:29.559559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e061d7b-ed96-42f7-a806-0a58c3faad28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:29.559956 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 297/588] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:29.560980 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e429f-c40c-467b-8cf1-6980c9d0cd2c None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:29.564822 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e061d7b-ed96-42f7-a806-0a58c3faad28 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:29.565233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e061d7b-ed96-42f7-a806-0a58c3faad28 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:29.565614 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e061d7b-ed96-42f7-a806-0a58c3faad28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:29.578356 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e061d7b-ed96-42f7-a806-0a58c3faad28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:29.579994 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e061d7b-ed96-42f7-a806-0a58c3faad28 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:29.601263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-9e061d7b-ed96-42f7-a806-0a58c3faad28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:29.601263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:29.601263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:29.601263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:29.601263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:29.601263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:29.601263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:29.601263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:29.601263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:29.601263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:29.601263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:29.601263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:29.601263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:29.601263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:29.601263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:29.602233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:29.602233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:29.602233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:29.602233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:29.602233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:29.602233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:29.602233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:29.602233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:29.602233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:29.602233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:29.602233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:29.602233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:29.602233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:29.602233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:29.602233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:29.602233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:29.602233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:29.603132 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:29.603132 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:29.603132 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:29.603132 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:29.603813 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 292/589] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:29.605107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e061d7b-ed96-42f7-a806-0a58c3faad28 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:29.667560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:29.695347 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e429f-c40c-467b-8cf1-6980c9d0cd2c None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:29.740340 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6181398-a6b1-4119-a486-a8cd6cd95f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:29.741476 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6181398-a6b1-4119-a486-a8cd6cd95f48 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:29.741826 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6181398-a6b1-4119-a486-a8cd6cd95f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:29.742172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6181398-a6b1-4119-a486-a8cd6cd95f48 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:29.743077 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6181398-a6b1-4119-a486-a8cd6cd95f48 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:29.743463 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e061d7b-ed96-42f7-a806-0a58c3faad28 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:29.743582 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6181398-a6b1-4119-a486-a8cd6cd95f48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:29.745048 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6181398-a6b1-4119-a486-a8cd6cd95f48 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:29.773800 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 298/590] 10.222.0.16 () {56 vars in 1137 bytes} [Mon May 23 12:19:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:29.773800 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6181398-a6b1-4119-a486-a8cd6cd95f48 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:29.798099 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1203d-1070-4f75-8a2a-c9e5db383bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:29.799635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1203d-1070-4f75-8a2a-c9e5db383bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:29.800095 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1203d-1070-4f75-8a2a-c9e5db383bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:29.800789 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1203d-1070-4f75-8a2a-c9e5db383bc7 None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:29.801923 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1203d-1070-4f75-8a2a-c9e5db383bc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:29.803875 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1203d-1070-4f75-8a2a-c9e5db383bc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:29.806032 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1203d-1070-4f75-8a2a-c9e5db383bc7 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:29.823422 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 293/591] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:29.824583 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1203d-1070-4f75-8a2a-c9e5db383bc7 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:29.829894 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:29.895110 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6181398-a6b1-4119-a486-a8cd6cd95f48 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:29.944905 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b0dc3a-9338-4058-86fb-971297b3182a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:29.946384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b0dc3a-9338-4058-86fb-971297b3182a None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:29.946923 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b0dc3a-9338-4058-86fb-971297b3182a None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:29.947352 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b0dc3a-9338-4058-86fb-971297b3182a None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:29.948506 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b0dc3a-9338-4058-86fb-971297b3182a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:29.949346 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b0dc3a-9338-4058-86fb-971297b3182a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:29.951425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b0dc3a-9338-4058-86fb-971297b3182a None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:29.955038 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1203d-1070-4f75-8a2a-c9e5db383bc7 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:29.958128 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:29.968816 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 299/592] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:29.970168 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b0dc3a-9338-4058-86fb-971297b3182a None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:29.995021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0245107-9b8d-41f9-8178-f82c56e8b025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:29.995795 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0245107-9b8d-41f9-8178-f82c56e8b025 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:29.996247 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0245107-9b8d-41f9-8178-f82c56e8b025 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:29.996614 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0245107-9b8d-41f9-8178-f82c56e8b025 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:30.010499 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0245107-9b8d-41f9-8178-f82c56e8b025 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:30.011951 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0245107-9b8d-41f9-8178-f82c56e8b025 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:30.039537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-c0245107-9b8d-41f9-8178-f82c56e8b025 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:30.039537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:30.039537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:30.039537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:30.039537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:30.039537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:30.039537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:30.039537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:30.039537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:30.039537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:30.039537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:30.039537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:30.039537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:30.039537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:30.039537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:30.040538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:30.040538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:30.040538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:30.040538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:30.040538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:30.040538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:30.040538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:30.040538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:30.040538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:30.040538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:30.040538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:30.040538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:30.040538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:30.040538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:30.040538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:30.040538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:30.040538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:30.041436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:30.041436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:30.041436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:30.041436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:30.045217 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 294/593] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:30.046090 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0245107-9b8d-41f9-8178-f82c56e8b025 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:30.113096 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b0dc3a-9338-4058-86fb-971297b3182a None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:30.123641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:30.156818 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0245107-9b8d-41f9-8178-f82c56e8b025 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:30.162364 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af120247-d58c-4cd1-9ac4-0a82dd625e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:30.163429 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af120247-d58c-4cd1-9ac4-0a82dd625e32 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:30.163916 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af120247-d58c-4cd1-9ac4-0a82dd625e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:30.164443 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af120247-d58c-4cd1-9ac4-0a82dd625e32 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:30.175690 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af120247-d58c-4cd1-9ac4-0a82dd625e32 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:30.177454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af120247-d58c-4cd1-9ac4-0a82dd625e32 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:30.193515 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07823c4b-98a1-4fe5-9a63-49665436d61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:30.194724 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07823c4b-98a1-4fe5-9a63-49665436d61c None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:30.195007 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07823c4b-98a1-4fe5-9a63-49665436d61c None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:30.195285 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07823c4b-98a1-4fe5-9a63-49665436d61c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:30.197749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-af120247-d58c-4cd1-9ac4-0a82dd625e32 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 23 12:19:30.197749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:30.197749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:30.197749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:30.197749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:30.197749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:30.197749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:30.197749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:30.197749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:30.197749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:30.197749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:30.197749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:30.197749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:19:30.197749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:19:30.197749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:19:30.197749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:19:30.198667 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:30.198667 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:30.198667 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:19:30.198667 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:30.198667 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:19:30.198667 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:30.198667 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:30.198667 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:30.198667 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:30.198667 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:30.198667 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:30.198667 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:30.198667 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:30.198667 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:30.198667 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:30.198667 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:30.198667 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:30.198667 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:30.199630 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:30.199630 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:30.199630 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:19:30.199630 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:19:30.199630 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:19:30.199630 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:19:30.199630 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:19:30.199630 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:19:30.199630 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 23 12:19:30.199630 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:30.202726 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 300/594] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 12:19:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:30.206339 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af120247-d58c-4cd1-9ac4-0a82dd625e32 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:30.214936 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07823c4b-98a1-4fe5-9a63-49665436d61c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:30.216497 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07823c4b-98a1-4fe5-9a63-49665436d61c None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:30.232683 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-07823c4b-98a1-4fe5-9a63-49665436d61c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:30.232683 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:30.232683 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:30.232683 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:30.232683 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:30.232683 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:30.232683 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:30.232683 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:30.232683 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:30.232683 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:30.232683 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:30.232683 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:30.232683 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:19:30.232683 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:19:30.232683 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:19:30.233625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:19:30.233625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:30.233625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:30.233625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:30.233625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:30.233625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:30.233625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:30.233625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:30.233625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:30.233625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:30.233625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:30.233625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:30.233625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:30.233625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:30.233625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:30.233625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:30.233625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:30.234532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:19:30.234532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:19:30.234532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:19:30.234532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:19:30.234532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:19:30.234532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:19:30.234532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:30.234532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:30.241301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 295/595] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:30.242004 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07823c4b-98a1-4fe5-9a63-49665436d61c None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:30.263053 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:30.346792 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af120247-d58c-4cd1-9ac4-0a82dd625e32 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:30.359788 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07823c4b-98a1-4fe5-9a63-49665436d61c None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:30.391398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbc46a-1186-47e8-82f8-6dd73d06781f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:30.392755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbc46a-1186-47e8-82f8-6dd73d06781f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:30.393482 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbc46a-1186-47e8-82f8-6dd73d06781f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:30.393960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbc46a-1186-47e8-82f8-6dd73d06781f None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:30.395050 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dbc46a-1186-47e8-82f8-6dd73d06781f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:30.395845 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dbc46a-1186-47e8-82f8-6dd73d06781f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:30.397933 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dbc46a-1186-47e8-82f8-6dd73d06781f None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:30.400520 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6e1dc-b61e-4a28-a7ca-e4c6b39649d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:30.401286 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6e1dc-b61e-4a28-a7ca-e4c6b39649d8 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:30.401637 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6e1dc-b61e-4a28-a7ca-e4c6b39649d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:30.401990 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6e1dc-b61e-4a28-a7ca-e4c6b39649d8 None admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:30.402912 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6e1dc-b61e-4a28-a7ca-e4c6b39649d8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:30.403299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6e1dc-b61e-4a28-a7ca-e4c6b39649d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:30.404854 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6e1dc-b61e-4a28-a7ca-e4c6b39649d8 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:30.416057 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 301/596] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:30.417167 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbc46a-1186-47e8-82f8-6dd73d06781f None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:30.432337 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 296/597] 10.222.0.16 () {56 vars in 1139 bytes} [Mon May 23 12:19:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:30.433649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6e1dc-b61e-4a28-a7ca-e4c6b39649d8 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:30.456185 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:30.533769 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:30.544336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbc46a-1186-47e8-82f8-6dd73d06781f None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:30.571031 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6e1dc-b61e-4a28-a7ca-e4c6b39649d8 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:30.584374 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd06d67-6c42-4042-a5b8-ebd4b1eae713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:30.585439 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd06d67-6c42-4042-a5b8-ebd4b1eae713 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:30.585948 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd06d67-6c42-4042-a5b8-ebd4b1eae713 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:30.586411 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd06d67-6c42-4042-a5b8-ebd4b1eae713 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:30.587967 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd06d67-6c42-4042-a5b8-ebd4b1eae713 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:30.588469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd06d67-6c42-4042-a5b8-ebd4b1eae713 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:30.590237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd06d67-6c42-4042-a5b8-ebd4b1eae713 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:30.608638 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d41e00-f9ee-4fcd-8061-7d535b2dce57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:30.608950 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 302/598] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:19:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:30.609700 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d41e00-f9ee-4fcd-8061-7d535b2dce57 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:30.610086 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d41e00-f9ee-4fcd-8061-7d535b2dce57 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:30.610374 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd06d67-6c42-4042-a5b8-ebd4b1eae713 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:30.610538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d41e00-f9ee-4fcd-8061-7d535b2dce57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:30.619003 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d41e00-f9ee-4fcd-8061-7d535b2dce57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:30.620641 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d41e00-f9ee-4fcd-8061-7d535b2dce57 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:30.635768 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-08d41e00-f9ee-4fcd-8061-7d535b2dce57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:30.635768 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:30.635768 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:30.635768 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:30.635768 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:30.635768 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:30.635768 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:30.635768 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:30.635768 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:30.635768 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:30.635768 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:30.635768 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:30.635768 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:19:30.635768 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:19:30.635768 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:19:30.636926 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:19:30.636926 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:30.636926 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:30.636926 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:30.636926 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:30.636926 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:30.636926 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:30.636926 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:30.636926 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:30.636926 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:30.636926 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:30.636926 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:30.636926 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:30.636926 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:30.636926 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:30.636926 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:30.636926 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:30.637853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:19:30.637853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:19:30.637853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:19:30.637853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:19:30.637853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:19:30.637853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:19:30.637853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:30.637853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:30.642663 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 297/599] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:30.643592 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d41e00-f9ee-4fcd-8061-7d535b2dce57 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:30.708143 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:30.727377 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd06d67-6c42-4042-a5b8-ebd4b1eae713 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:30.757080 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d41e00-f9ee-4fcd-8061-7d535b2dce57 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:30.765830 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8574cd0-fd92-4c92-97d0-93020472dc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:30.766923 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8574cd0-fd92-4c92-97d0-93020472dc9b None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:30.767457 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8574cd0-fd92-4c92-97d0-93020472dc9b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:30.768001 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8574cd0-fd92-4c92-97d0-93020472dc9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:30.783329 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8574cd0-fd92-4c92-97d0-93020472dc9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:30.786120 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8574cd0-fd92-4c92-97d0-93020472dc9b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:30.793583 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8caf18c-47e6-4556-a950-006b4ed90e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:30.794540 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8caf18c-47e6-4556-a950-006b4ed90e40 None admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:30.794827 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8caf18c-47e6-4556-a950-006b4ed90e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:30.795103 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8caf18c-47e6-4556-a950-006b4ed90e40 None admin] PATH_INFO: `/v3/projects/839585501f414e1ab02501422510e54f/users/b83535b443a84275a160cd84cb3ad5a0/roles/81d82471e2be4f0aa8164b75c16a9718` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:30.797996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:30.812679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-a8574cd0-fd92-4c92-97d0-93020472dc9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:30.812679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:30.812679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:30.812679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:30.812679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:30.812679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:30.812679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:30.812679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:30.812679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:30.812679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:30.812679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:30.812679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:30.812679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:30.812679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:30.812679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:30.813608 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:30.813608 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:30.813608 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:30.813608 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:30.813608 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:30.813608 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:30.813608 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:30.813608 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:30.813608 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:30.813608 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:30.813608 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:30.813608 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:30.813608 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:30.813608 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:30.813608 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:30.813608 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:30.813608 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:30.814534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:30.814534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:30.814534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:30.814534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:30.814795 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8caf18c-47e6-4556-a950-006b4ed90e40 None admin] RBAC: Authorizing `identity:create_grant(project_id=839585501f414e1ab02501422510e54f, user_id=b83535b443a84275a160cd84cb3ad5a0, role_id=81d82471e2be4f0aa8164b75c16a9718)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:30.814795 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8caf18c-47e6-4556-a950-006b4ed90e40 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:30.817306 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 303/600] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:30.819258 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8574cd0-fd92-4c92-97d0-93020472dc9b None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:30.834143 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 298/601] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 12:19:30 2022] PUT /identity/v3/projects/839585501f414e1ab02501422510e54f/users/b83535b443a84275a160cd84cb3ad5a0/roles/81d82471e2be4f0aa8164b75c16a9718 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:19:30.834917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8caf18c-47e6-4556-a950-006b4ed90e40 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:30.971512 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8574cd0-fd92-4c92-97d0-93020472dc9b None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:31.032063 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6681e-b2e3-4144-b4e9-52a4803dd7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:31.033183 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6681e-b2e3-4144-b4e9-52a4803dd7b3 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:31.033865 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6681e-b2e3-4144-b4e9-52a4803dd7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:31.034543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6681e-b2e3-4144-b4e9-52a4803dd7b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:31.035838 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f6681e-b2e3-4144-b4e9-52a4803dd7b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:31.037483 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f6681e-b2e3-4144-b4e9-52a4803dd7b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:31.039238 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f6681e-b2e3-4144-b4e9-52a4803dd7b3 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:31.050557 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:31.061211 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 304/602] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:19:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:31.070482 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6681e-b2e3-4144-b4e9-52a4803dd7b3 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:31.081421 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8caf18c-47e6-4556-a950-006b4ed90e40 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:31.122770 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0eafcd-99e5-47f8-8ae8-f016ed60e59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:31.123781 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0eafcd-99e5-47f8-8ae8-f016ed60e59a None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:31.124293 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0eafcd-99e5-47f8-8ae8-f016ed60e59a None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:31.124773 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0eafcd-99e5-47f8-8ae8-f016ed60e59a None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:31.126254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0eafcd-99e5-47f8-8ae8-f016ed60e59a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:31.126764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0eafcd-99e5-47f8-8ae8-f016ed60e59a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:31.128839 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0eafcd-99e5-47f8-8ae8-f016ed60e59a None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:31.145590 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 299/603] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:31.147905 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff3a9d-b428-4050-a49a-894c0e5e5e43 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:31.148429 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff3a9d-b428-4050-a49a-894c0e5e5e43 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:31.148931 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff3a9d-b428-4050-a49a-894c0e5e5e43 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:31.150781 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 300/604] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:31.156346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff3a9d-b428-4050-a49a-894c0e5e5e43 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:31.201813 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6681e-b2e3-4144-b4e9-52a4803dd7b3 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:31.229292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:31.242812 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2398d5a-875a-4ae6-9d6f-66a4e8683232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:31.243848 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2398d5a-875a-4ae6-9d6f-66a4e8683232 None admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:31.244138 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2398d5a-875a-4ae6-9d6f-66a4e8683232 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:31.244485 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2398d5a-875a-4ae6-9d6f-66a4e8683232 None admin] PATH_INFO: `/v3/projects/839585501f414e1ab02501422510e54f/users/e12ef3e115544a51a6bf9b08d265fefb/roles/81d82471e2be4f0aa8164b75c16a9718` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:31.260923 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2398d5a-875a-4ae6-9d6f-66a4e8683232 None admin] RBAC: Authorizing `identity:create_grant(project_id=839585501f414e1ab02501422510e54f, user_id=e12ef3e115544a51a6bf9b08d265fefb, role_id=81d82471e2be4f0aa8164b75c16a9718)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:31.262528 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2398d5a-875a-4ae6-9d6f-66a4e8683232 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:31.280330 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 305/605] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 12:19:31 2022] PUT /identity/v3/projects/839585501f414e1ab02501422510e54f/users/e12ef3e115544a51a6bf9b08d265fefb/roles/81d82471e2be4f0aa8164b75c16a9718 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:19:31.290305 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545e5b3-1429-4232-8597-bb804cdaf265 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:31.290640 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545e5b3-1429-4232-8597-bb804cdaf265 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:31.290983 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f545e5b3-1429-4232-8597-bb804cdaf265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:31.299864 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff3a9d-b428-4050-a49a-894c0e5e5e43 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:31.321481 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-f545e5b3-1429-4232-8597-bb804cdaf265 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:31.323157 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-f545e5b3-1429-4232-8597-bb804cdaf265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:31.342566 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3532d26b-b7af-4d73-bf5c-acd9feced509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:31.343841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3532d26b-b7af-4d73-bf5c-acd9feced509 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:31.344743 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3532d26b-b7af-4d73-bf5c-acd9feced509 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:31.346212 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3532d26b-b7af-4d73-bf5c-acd9feced509 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:31.359329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3532d26b-b7af-4d73-bf5c-acd9feced509 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:31.363054 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3532d26b-b7af-4d73-bf5c-acd9feced509 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:31.386375 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-3532d26b-b7af-4d73-bf5c-acd9feced509 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:31.386375 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:31.386375 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:31.386375 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:31.386375 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:31.386375 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:31.386375 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:31.386375 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:31.386375 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:31.386375 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:31.386375 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:31.386375 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:31.386375 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:19:31.386375 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:19:31.386375 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:19:31.387379 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:19:31.387379 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:31.387379 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:31.387379 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:31.387379 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:31.387379 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:31.387379 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:31.387379 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:31.387379 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:31.387379 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:31.387379 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:31.387379 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:31.387379 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:31.387379 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:31.387379 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:31.387379 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:31.387379 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:31.388360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:19:31.388360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:19:31.388360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:19:31.388360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:19:31.388360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:19:31.388360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:19:31.388360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:31.388360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:31.391005 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 301/606] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:31.397689 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3532d26b-b7af-4d73-bf5c-acd9feced509 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:31.411361 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:31.455661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 306/607] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:31.487779 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f545e5b3-1429-4232-8597-bb804cdaf265 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:31.543827 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3532d26b-b7af-4d73-bf5c-acd9feced509 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:31.555798 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:31.589692 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147423c3-3936-4292-a435-438b2b1fc1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:31.590271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147423c3-3936-4292-a435-438b2b1fc1fe None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:31.590565 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147423c3-3936-4292-a435-438b2b1fc1fe None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:31.590849 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147423c3-3936-4292-a435-438b2b1fc1fe None admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:31.591647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147423c3-3936-4292-a435-438b2b1fc1fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:31.591967 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147423c3-3936-4292-a435-438b2b1fc1fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:31.593661 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147423c3-3936-4292-a435-438b2b1fc1fe None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:31.610280 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 302/608] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:19:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:19:31.612329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0263ad3-8f2b-4113-a94c-ddc26dfc5d65 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:31.612797 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0263ad3-8f2b-4113-a94c-ddc26dfc5d65 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:31.612935 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0263ad3-8f2b-4113-a94c-ddc26dfc5d65 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:31.614272 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 303/609] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:31.620110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0263ad3-8f2b-4113-a94c-ddc26dfc5d65 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:31.656158 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f545e5b3-1429-4232-8597-bb804cdaf265 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:31.699119 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b80d82-9cfd-4021-82e7-2318c1715a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:31.699913 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b80d82-9cfd-4021-82e7-2318c1715a7f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:31.700293 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b80d82-9cfd-4021-82e7-2318c1715a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:31.700693 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b80d82-9cfd-4021-82e7-2318c1715a7f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:31.713070 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b80d82-9cfd-4021-82e7-2318c1715a7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:31.721426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-93b80d82-9cfd-4021-82e7-2318c1715a7f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:31.721426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:31.721426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:31.721426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:31.721426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:31.721426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:31.721426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:31.721426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:31.721426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:31.721426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:31.721426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:31.721426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:31.721426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:31.721426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:31.721426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:31.721426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:31.722717 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:31.722717 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:31.722717 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:31.722717 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:31.722717 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:31.722717 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:31.722717 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:31.722717 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:31.722717 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:31.722717 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:31.722717 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:31.722717 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:31.722717 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:31.722717 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:31.722717 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:31.722717 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:31.723758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:31.723758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:31.723758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:31.723758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:31.723758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:31.723758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:31.723758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:31.723758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:31.723758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:31.723758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:31.723758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:31.723758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:31.725642 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 307/610] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 12:19:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:19:31.727177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147df17-9004-4d84-bda8-e11de2338dd7 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:31.727533 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147df17-9004-4d84-bda8-e11de2338dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:31.727883 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147df17-9004-4d84-bda8-e11de2338dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:31.747485 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-5147df17-9004-4d84-bda8-e11de2338dd7 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:31.748778 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5147df17-9004-4d84-bda8-e11de2338dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:31.755414 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0263ad3-8f2b-4113-a94c-ddc26dfc5d65 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:31.784564 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:31.803368 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d34ed-c911-4a2b-a0cd-ef4ec25405a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:31.804798 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d34ed-c911-4a2b-a0cd-ef4ec25405a0 None admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:31.805258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d34ed-c911-4a2b-a0cd-ef4ec25405a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:31.805856 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d34ed-c911-4a2b-a0cd-ef4ec25405a0 None admin] PATH_INFO: `/v3/projects/839585501f414e1ab02501422510e54f/users/b6677be3ddf647e2b971bb3ba2c34a3a/roles/81d82471e2be4f0aa8164b75c16a9718` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:31.827036 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16d34ed-c911-4a2b-a0cd-ef4ec25405a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=839585501f414e1ab02501422510e54f, user_id=b6677be3ddf647e2b971bb3ba2c34a3a, role_id=81d82471e2be4f0aa8164b75c16a9718)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:31.829133 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16d34ed-c911-4a2b-a0cd-ef4ec25405a0 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:31.836204 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 308/611] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:31.836204 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5147df17-9004-4d84-bda8-e11de2338dd7 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:31.858484 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 304/612] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 12:19:31 2022] PUT /identity/v3/projects/839585501f414e1ab02501422510e54f/users/b6677be3ddf647e2b971bb3ba2c34a3a/roles/81d82471e2be4f0aa8164b75c16a9718 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:19:31.865554 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d34ed-c911-4a2b-a0cd-ef4ec25405a0 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:31.899548 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:31.974289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5147df17-9004-4d84-bda8-e11de2338dd7 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:32.013430 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4ca92-5853-448c-8e11-1cb704c33e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:32.014508 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4ca92-5853-448c-8e11-1cb704c33e96 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:32.015001 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4ca92-5853-448c-8e11-1cb704c33e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:32.015492 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4ca92-5853-448c-8e11-1cb704c33e96 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:32.016526 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b4ca92-5853-448c-8e11-1cb704c33e96 None admin] RBAC: Adding query filter params (name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:32.017065 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b4ca92-5853-448c-8e11-1cb704c33e96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:32.018698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b4ca92-5853-448c-8e11-1cb704c33e96 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:32.028798 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 309/613] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 12:19:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:32.034161 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4ca92-5853-448c-8e11-1cb704c33e96 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:32.075116 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:32.123795 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d34ed-c911-4a2b-a0cd-ef4ec25405a0 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:32.171071 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4ca92-5853-448c-8e11-1cb704c33e96 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:32.171286 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc0870-3016-4a28-a24f-88d8fef9152c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:32.171875 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc0870-3016-4a28-a24f-88d8fef9152c None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:32.172155 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc0870-3016-4a28-a24f-88d8fef9152c None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:32.172470 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc0870-3016-4a28-a24f-88d8fef9152c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:32.191111 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc0870-3016-4a28-a24f-88d8fef9152c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:32.202562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-59fc0870-3016-4a28-a24f-88d8fef9152c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:32.202562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:32.202562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:32.202562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:32.202562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:32.202562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:32.202562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:32.202562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:32.202562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:32.202562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:32.202562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:32.202562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:32.202562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:32.202562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:32.202562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:32.202562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:32.203566 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:32.203566 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:32.203566 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:32.203566 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:32.203566 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:32.203566 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:32.203566 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:32.203566 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:32.203566 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:32.203566 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:32.203566 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:32.203566 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:32.203566 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:32.203566 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:32.203566 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:32.203566 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:32.204448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:32.204448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:32.204448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:32.204448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:32.204448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:32.204448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:32.204448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:32.204448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:32.204448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:32.204448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:32.204448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:32.204448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:32.204448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 305/614] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 12:19:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:32.218488 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3c53b-8ccc-46d5-8666-e1dbbf86f6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:32.219380 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc0870-3016-4a28-a24f-88d8fef9152c None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:32.220005 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3c53b-8ccc-46d5-8666-e1dbbf86f6ed None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:32.220511 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3c53b-8ccc-46d5-8666-e1dbbf86f6ed None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:32.221009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3c53b-8ccc-46d5-8666-e1dbbf86f6ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:32.237521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f3c53b-8ccc-46d5-8666-e1dbbf86f6ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:32.240554 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f3c53b-8ccc-46d5-8666-e1dbbf86f6ed None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:32.261705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-10f3c53b-8ccc-46d5-8666-e1dbbf86f6ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:32.261705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:32.261705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:32.261705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:32.261705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:32.261705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:32.261705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:32.261705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:32.261705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:32.261705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:32.261705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:32.261705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:32.261705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:32.261705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:32.261705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:32.262721 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:32.262721 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:32.262721 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:32.262721 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:32.262721 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:32.262721 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:32.262721 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:32.262721 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:32.262721 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:32.262721 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:32.262721 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:32.262721 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:32.262721 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:32.262721 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:32.262721 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:32.262721 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:32.262721 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:32.263615 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:32.263615 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:32.263615 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:32.263615 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:32.266428 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 310/615] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:19:32.271983 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3c53b-8ccc-46d5-8666-e1dbbf86f6ed None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:32.334330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:32.360053 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc0870-3016-4a28-a24f-88d8fef9152c None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:32.409076 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff3daf-2fbe-48b6-90de-1504a1be42b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:32.409739 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff3daf-2fbe-48b6-90de-1504a1be42b9 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:32.410014 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff3daf-2fbe-48b6-90de-1504a1be42b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:32.410293 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff3daf-2fbe-48b6-90de-1504a1be42b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:32.411093 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff3daf-2fbe-48b6-90de-1504a1be42b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:32.411343 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff3daf-2fbe-48b6-90de-1504a1be42b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:32.416211 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff3daf-2fbe-48b6-90de-1504a1be42b9 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:32.422163 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 306/616] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 12:19:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:32.428014 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff3daf-2fbe-48b6-90de-1504a1be42b9 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:32.456978 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:32.493159 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3c53b-8ccc-46d5-8666-e1dbbf86f6ed None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:32.547879 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff3daf-2fbe-48b6-90de-1504a1be42b9 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:32.580893 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde1c20-cfe7-43bf-8cb6-20fcd108eda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:32.581578 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde1c20-cfe7-43bf-8cb6-20fcd108eda5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:32.581869 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde1c20-cfe7-43bf-8cb6-20fcd108eda5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:32.582155 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fde1c20-cfe7-43bf-8cb6-20fcd108eda5 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:32.582898 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fde1c20-cfe7-43bf-8cb6-20fcd108eda5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:32.583215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fde1c20-cfe7-43bf-8cb6-20fcd108eda5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:32.584842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fde1c20-cfe7-43bf-8cb6-20fcd108eda5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:32.594616 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1e14e-73d8-4ffb-9e03-6f03b3bc3deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:32.595944 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1e14e-73d8-4ffb-9e03-6f03b3bc3deb None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:32.596448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1e14e-73d8-4ffb-9e03-6f03b3bc3deb None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:32.596827 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1e14e-73d8-4ffb-9e03-6f03b3bc3deb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:32.597792 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 311/617] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:32.603666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde1c20-cfe7-43bf-8cb6-20fcd108eda5 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:32.616546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:32.621855 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a1e14e-73d8-4ffb-9e03-6f03b3bc3deb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:32.623397 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a1e14e-73d8-4ffb-9e03-6f03b3bc3deb None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:32.663619 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-e4a1e14e-73d8-4ffb-9e03-6f03b3bc3deb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:32.663619 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:32.663619 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:32.663619 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:32.663619 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:32.663619 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:32.663619 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:32.663619 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:32.663619 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:32.663619 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:32.663619 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:32.663619 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:32.663619 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:32.663619 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:32.663619 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:32.664623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:32.664623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:32.664623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:32.664623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:32.664623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:32.664623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:32.664623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:32.664623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:32.664623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:32.664623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:32.664623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:32.664623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:32.664623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:32.664623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:32.664623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:32.664623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:32.664623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:32.665518 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:32.665518 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:32.665518 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:32.665518 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:32.667946 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 307/618] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:19:32.677141 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1e14e-73d8-4ffb-9e03-6f03b3bc3deb None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:32.707744 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fde1c20-cfe7-43bf-8cb6-20fcd108eda5 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:32.746827 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba04300-aa63-4518-b3c5-12455824dce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:32.747629 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba04300-aa63-4518-b3c5-12455824dce1 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:32.747914 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba04300-aa63-4518-b3c5-12455824dce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:32.748204 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba04300-aa63-4518-b3c5-12455824dce1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:32.761528 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba04300-aa63-4518-b3c5-12455824dce1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:32.762990 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba04300-aa63-4518-b3c5-12455824dce1 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:32.793067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-8ba04300-aa63-4518-b3c5-12455824dce1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 12:19:32.793067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:32.793067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:32.793067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:32.793067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:32.793067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:32.793067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:32.793067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:32.793067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:32.793067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:32.793067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:32.793067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:32.793067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:19:32.793067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:19:32.793067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:19:32.793067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:19:32.794179 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:32.794179 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:32.794179 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:19:32.794179 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:32.794179 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:19:32.794179 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:32.794179 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:32.794179 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:32.794179 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:32.794179 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:32.794179 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:32.794179 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:32.794179 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:32.794179 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:32.794179 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:32.794179 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:32.794179 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:32.794179 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:32.795138 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:32.795138 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:32.795138 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:19:32.795138 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:19:32.795138 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:19:32.795138 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:19:32.795138 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:19:32.795138 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:19:32.795138 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 12:19:32.795138 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:32.803441 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 312/619] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:19:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:19:32.809209 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba04300-aa63-4518-b3c5-12455824dce1 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:32.815480 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:32.899155 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1e14e-73d8-4ffb-9e03-6f03b3bc3deb None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:32.966905 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd84ea50-52dd-4180-95e8-834528fba7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:32.977046 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84ea50-52dd-4180-95e8-834528fba7ea None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:32.977345 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84ea50-52dd-4180-95e8-834528fba7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:32.977584 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84ea50-52dd-4180-95e8-834528fba7ea None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:32.978337 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd84ea50-52dd-4180-95e8-834528fba7ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:32.978699 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd84ea50-52dd-4180-95e8-834528fba7ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:32.980051 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd84ea50-52dd-4180-95e8-834528fba7ea None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:33.001664 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba04300-aa63-4518-b3c5-12455824dce1 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:33.005054 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 308/620] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:33.005584 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd84ea50-52dd-4180-95e8-834528fba7ea None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:33.040646 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d965efb-62bf-4b86-b025-32ef17dbdb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:33.041677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d965efb-62bf-4b86-b025-32ef17dbdb1d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:33.042179 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d965efb-62bf-4b86-b025-32ef17dbdb1d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:33.042671 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d965efb-62bf-4b86-b025-32ef17dbdb1d None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:33.043827 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d965efb-62bf-4b86-b025-32ef17dbdb1d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:33.044374 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d965efb-62bf-4b86-b025-32ef17dbdb1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:33.046094 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d965efb-62bf-4b86-b025-32ef17dbdb1d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:33.065938 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 313/621] 10.222.0.16 () {56 vars in 1135 bytes} [Mon May 23 12:19:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:33.073137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d965efb-62bf-4b86-b025-32ef17dbdb1d None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:33.089900 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:33.143467 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd84ea50-52dd-4180-95e8-834528fba7ea None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:33.188086 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f4cb1-a5e3-453d-a16e-ca7b9a94eae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:33.188698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:33.188901 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f4cb1-a5e3-453d-a16e-ca7b9a94eae2 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:33.189194 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f4cb1-a5e3-453d-a16e-ca7b9a94eae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:33.189481 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f4cb1-a5e3-453d-a16e-ca7b9a94eae2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:33.207753 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2f4cb1-a5e3-453d-a16e-ca7b9a94eae2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:33.209810 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2f4cb1-a5e3-453d-a16e-ca7b9a94eae2 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:33.240228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-af2f4cb1-a5e3-453d-a16e-ca7b9a94eae2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 23 12:19:33.240228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:33.240228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:33.240228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:33.240228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:33.240228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:33.240228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:33.240228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:33.240228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:33.240228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:33.240228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:33.240228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:33.240228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:19:33.240228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:19:33.240228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:19:33.240228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:19:33.241274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:33.241274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:33.241274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:19:33.241274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:33.241274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:19:33.241274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:33.241274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:33.241274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:33.241274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:33.241274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:33.241274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:33.241274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:33.241274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:33.241274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:33.241274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:33.241274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:33.241274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:33.241274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:33.242237 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:33.242237 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:33.242237 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:19:33.242237 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:19:33.242237 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:19:33.242237 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:19:33.242237 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:19:33.242237 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:19:33.242237 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 23 12:19:33.242237 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:33.254414 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 309/622] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:19:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:33.261872 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f4cb1-a5e3-453d-a16e-ca7b9a94eae2 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:33.281709 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d965efb-62bf-4b86-b025-32ef17dbdb1d None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:33.357073 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af137293-d287-4e9c-80c5-d5eb09d5646c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:33.358012 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af137293-d287-4e9c-80c5-d5eb09d5646c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:33.358305 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af137293-d287-4e9c-80c5-d5eb09d5646c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:33.358585 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af137293-d287-4e9c-80c5-d5eb09d5646c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:33.388972 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af137293-d287-4e9c-80c5-d5eb09d5646c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:33.390431 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af137293-d287-4e9c-80c5-d5eb09d5646c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:33.421580 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-af137293-d287-4e9c-80c5-d5eb09d5646c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:33.421580 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:33.421580 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:33.421580 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:33.421580 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:33.421580 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:33.421580 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:33.421580 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:33.421580 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:33.421580 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:33.421580 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:33.421580 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:33.421580 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:33.421580 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:33.421580 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:33.422568 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:33.422568 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:33.422568 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:33.422568 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:33.422568 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:33.422568 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:33.422568 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:33.422568 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:33.422568 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:33.422568 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:33.422568 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:33.422568 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:33.422568 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:33.422568 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:33.422568 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:33.422568 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:33.422568 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:33.423449 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:33.423449 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:33.423449 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:33.423449 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:33.423686 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 314/623] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:33.425112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc3416-be70-4115-9bb1-7844b3d20088 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:33.425381 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc3416-be70-4115-9bb1-7844b3d20088 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:33.425660 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc3416-be70-4115-9bb1-7844b3d20088 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:33.426975 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 315/624] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:33.433603 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbc3416-be70-4115-9bb1-7844b3d20088 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:33.441203 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f4cb1-a5e3-453d-a16e-ca7b9a94eae2 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:33.453987 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:33.483039 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307fb9ea-7497-47f8-b2e6-84d4bd6641f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:33.484147 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307fb9ea-7497-47f8-b2e6-84d4bd6641f1 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:33.484660 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307fb9ea-7497-47f8-b2e6-84d4bd6641f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:33.485161 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307fb9ea-7497-47f8-b2e6-84d4bd6641f1 None admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:33.486249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307fb9ea-7497-47f8-b2e6-84d4bd6641f1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:33.486874 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307fb9ea-7497-47f8-b2e6-84d4bd6641f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:33.488656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307fb9ea-7497-47f8-b2e6-84d4bd6641f1 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:33.509895 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 310/625] 10.222.0.16 () {56 vars in 1137 bytes} [Mon May 23 12:19:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:33.511559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a8043-92c7-4d69-aab7-f8290fb81d42 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:33.512052 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a8043-92c7-4d69-aab7-f8290fb81d42 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:33.512584 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a8043-92c7-4d69-aab7-f8290fb81d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:33.530744 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-1b6a8043-92c7-4d69-aab7-f8290fb81d42 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:33.532484 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-1b6a8043-92c7-4d69-aab7-f8290fb81d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:33.564565 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbc3416-be70-4115-9bb1-7844b3d20088 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:33.582094 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:33.608730 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7cc125-8401-4b08-90b7-035a27c82dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:33.609438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7cc125-8401-4b08-90b7-035a27c82dee None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:33.609720 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7cc125-8401-4b08-90b7-035a27c82dee None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:33.609997 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7cc125-8401-4b08-90b7-035a27c82dee None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:33.612215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7cc125-8401-4b08-90b7-035a27c82dee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:33.612215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7cc125-8401-4b08-90b7-035a27c82dee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:33.612441 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7cc125-8401-4b08-90b7-035a27c82dee None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:33.616584 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 311/626] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:33.617501 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a8043-92c7-4d69-aab7-f8290fb81d42 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:33.626195 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 316/627] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:33.633710 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7cc125-8401-4b08-90b7-035a27c82dee None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:33.738149 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7cc125-8401-4b08-90b7-035a27c82dee None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:33.750547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:33.759447 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a8043-92c7-4d69-aab7-f8290fb81d42 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:33.777894 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9d7df-a8f5-415d-8e1e-a3dd46b84e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:33.778694 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9d7df-a8f5-415d-8e1e-a3dd46b84e90 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:33.779080 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9d7df-a8f5-415d-8e1e-a3dd46b84e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:33.779366 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9d7df-a8f5-415d-8e1e-a3dd46b84e90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:33.787424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9d7df-a8f5-415d-8e1e-a3dd46b84e90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:33.788945 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9d7df-a8f5-415d-8e1e-a3dd46b84e90 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:33.798212 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b775ce-6e23-4ec4-af62-23052b175713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:33.798864 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b775ce-6e23-4ec4-af62-23052b175713 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:33.799147 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b775ce-6e23-4ec4-af62-23052b175713 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:33.799425 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b775ce-6e23-4ec4-af62-23052b175713 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:33.805218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-c3f9d7df-a8f5-415d-8e1e-a3dd46b84e90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:33.805218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:33.805218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:33.805218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:33.805218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:33.805218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:33.805218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:33.805218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:33.805218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:33.805218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:33.805218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:33.805218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:33.805218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:19:33.805218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:19:33.805218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:19:33.806137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:19:33.806137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:33.806137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:33.806137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:33.806137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:33.806137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:33.806137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:33.806137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:33.806137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:33.806137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:33.806137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:33.806137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:33.806137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:33.806137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:33.806137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:33.806137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:33.806137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:33.807115 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:19:33.807115 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:19:33.807115 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:19:33.807115 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:19:33.807115 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:19:33.807115 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:19:33.807115 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:33.807115 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:33.820196 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 317/628] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:33.820196 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9d7df-a8f5-415d-8e1e-a3dd46b84e90 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:33.848243 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:33.880057 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b775ce-6e23-4ec4-af62-23052b175713 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:33.881613 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b775ce-6e23-4ec4-af62-23052b175713 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:33.928578 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9d7df-a8f5-415d-8e1e-a3dd46b84e90 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:33.965644 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060ec1b-5729-413e-a843-4b9301182da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:33.966438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060ec1b-5729-413e-a843-4b9301182da2 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:33.966719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060ec1b-5729-413e-a843-4b9301182da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:33.966991 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060ec1b-5729-413e-a843-4b9301182da2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:33.977559 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d060ec1b-5729-413e-a843-4b9301182da2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:33.985187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-d060ec1b-5729-413e-a843-4b9301182da2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:33.985187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:33.985187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:33.985187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:33.985187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:33.985187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:33.985187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:33.985187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:33.985187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:33.985187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:33.985187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:33.985187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:33.985187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:33.985187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:33.985187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:33.985187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:33.986317 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:33.986317 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:33.986317 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:33.986317 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:33.986317 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:33.986317 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:33.986317 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:33.986317 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:33.986317 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:33.986317 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:33.986317 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:33.986317 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:33.986317 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:33.986317 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:33.986317 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:33.986317 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:33.987184 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:33.987184 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:33.987184 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:33.987184 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:33.987184 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:33.987184 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:33.987184 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:33.987184 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:33.987184 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:33.987184 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:33.987184 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:33.987184 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:33.987184 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 318/629] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 12:19:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:33.987184 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060ec1b-5729-413e-a843-4b9301182da2 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:34.008021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:34.019878 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 312/630] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:19:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:19:34.021602 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717c432-9dac-4717-bf0c-e4d4de1b3974 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:34.022041 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717c432-9dac-4717-bf0c-e4d4de1b3974 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:34.022326 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717c432-9dac-4717-bf0c-e4d4de1b3974 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:34.027688 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 313/631] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:34.027688 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b717c432-9dac-4717-bf0c-e4d4de1b3974 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:34.095593 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:34.105241 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060ec1b-5729-413e-a843-4b9301182da2 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:34.139495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b717c432-9dac-4717-bf0c-e4d4de1b3974 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:34.141884 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89ef6a-2ecb-420d-b142-b6d7ca966bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:34.142547 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89ef6a-2ecb-420d-b142-b6d7ca966bfa None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:34.142836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89ef6a-2ecb-420d-b142-b6d7ca966bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:34.143117 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89ef6a-2ecb-420d-b142-b6d7ca966bfa None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:34.143904 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d89ef6a-2ecb-420d-b142-b6d7ca966bfa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:34.144243 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d89ef6a-2ecb-420d-b142-b6d7ca966bfa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:34.145685 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d89ef6a-2ecb-420d-b142-b6d7ca966bfa None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:34.164270 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 319/632] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:19:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:34.164270 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89ef6a-2ecb-420d-b142-b6d7ca966bfa None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:34.181353 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff627d68-c88b-410e-9a42-fa9b6f01bc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:34.182031 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff627d68-c88b-410e-9a42-fa9b6f01bc5b None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:34.182317 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff627d68-c88b-410e-9a42-fa9b6f01bc5b None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:34.182596 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff627d68-c88b-410e-9a42-fa9b6f01bc5b None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:34.183347 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff627d68-c88b-410e-9a42-fa9b6f01bc5b None admin] RBAC: Adding query filter params (name=service) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:34.183654 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff627d68-c88b-410e-9a42-fa9b6f01bc5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:34.185089 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff627d68-c88b-410e-9a42-fa9b6f01bc5b None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:34.193542 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 314/633] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 12:19:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:34.194757 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a8e99-3674-4b78-9311-40199f201167 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:34.195220 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a8e99-3674-4b78-9311-40199f201167 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:34.195512 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a8e99-3674-4b78-9311-40199f201167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:34.212443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-6b9a8e99-3674-4b78-9311-40199f201167 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:34.213642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-6b9a8e99-3674-4b78-9311-40199f201167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:34.276116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:34.276435 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89ef6a-2ecb-420d-b142-b6d7ca966bfa None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:34.295562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 315/634] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:34.296788 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9a8e99-3674-4b78-9311-40199f201167 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:34.320775 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d0bd7-8c1d-4103-9045-03e07848f3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:34.321605 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d0bd7-8c1d-4103-9045-03e07848f3f8 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:34.321920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d0bd7-8c1d-4103-9045-03e07848f3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:34.322315 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d0bd7-8c1d-4103-9045-03e07848f3f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:34.332799 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d0bd7-8c1d-4103-9045-03e07848f3f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:34.334814 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d0bd7-8c1d-4103-9045-03e07848f3f8 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:34.353122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-c82d0bd7-8c1d-4103-9045-03e07848f3f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:34.353122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:34.353122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:34.353122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:34.353122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:34.353122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:34.353122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:34.353122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:34.353122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:34.353122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:34.353122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:34.353122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:34.353122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:34.353122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:34.353122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:34.354091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:34.354091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:34.354091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:34.354091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:34.354091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:34.354091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:34.354091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:34.354091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:34.354091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:34.354091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:34.354091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:34.354091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:34.354091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:34.354091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:34.354091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:34.354091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:34.354091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:34.355460 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:34.355460 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:34.355460 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:34.355460 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:34.355460 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 320/635] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:34.355460 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d0bd7-8c1d-4103-9045-03e07848f3f8 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:34.360341 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:34.426339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9a8e99-3674-4b78-9311-40199f201167 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:34.462411 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd864246-515c-46fa-9e8f-40b2ae2f7849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:34.463307 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd864246-515c-46fa-9e8f-40b2ae2f7849 None admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:34.463589 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd864246-515c-46fa-9e8f-40b2ae2f7849 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:34.463868 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd864246-515c-46fa-9e8f-40b2ae2f7849 None admin] PATH_INFO: `/v3/projects/839585501f414e1ab02501422510e54f/users/71fb9a79ac494635804b93d4a0048205/roles/81d82471e2be4f0aa8164b75c16a9718` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:34.466655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d0bd7-8c1d-4103-9045-03e07848f3f8 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:34.479429 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd864246-515c-46fa-9e8f-40b2ae2f7849 None admin] RBAC: Authorizing `identity:create_grant(project_id=839585501f414e1ab02501422510e54f, user_id=71fb9a79ac494635804b93d4a0048205, role_id=81d82471e2be4f0aa8164b75c16a9718)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:34.480916 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd864246-515c-46fa-9e8f-40b2ae2f7849 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:34.496370 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 316/636] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 12:19:34 2022] PUT /identity/v3/projects/839585501f414e1ab02501422510e54f/users/71fb9a79ac494635804b93d4a0048205/roles/81d82471e2be4f0aa8164b75c16a9718 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:19:34.497663 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd864246-515c-46fa-9e8f-40b2ae2f7849 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:34.520638 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be4665-57cd-4943-9e2d-46853b1752bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:34.522927 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be4665-57cd-4943-9e2d-46853b1752bd None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:34.525408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be4665-57cd-4943-9e2d-46853b1752bd None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:34.526911 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be4665-57cd-4943-9e2d-46853b1752bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:34.546806 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8be4665-57cd-4943-9e2d-46853b1752bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:34.549180 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8be4665-57cd-4943-9e2d-46853b1752bd None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:34.560460 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:34.573145 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-b8be4665-57cd-4943-9e2d-46853b1752bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:34.573145 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:34.573145 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:34.573145 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:34.573145 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:34.573145 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:34.573145 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:34.573145 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:34.573145 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:34.573145 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:34.573145 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:34.573145 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:34.573145 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:34.573145 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:34.573145 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:34.574648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:34.574648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:34.574648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:34.574648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:34.574648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:34.574648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:34.574648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:34.574648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:34.574648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:34.574648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:34.574648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:34.574648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:34.574648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:34.574648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:34.574648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:34.574648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:34.574648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:34.575728 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:34.575728 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:34.575728 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:34.575728 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:34.575728 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 321/637] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:34.575728 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be4665-57cd-4943-9e2d-46853b1752bd None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:34.639450 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd864246-515c-46fa-9e8f-40b2ae2f7849 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:34.640043 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:34.679171 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ba81a-798c-432e-9d44-6d2e71dcb6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:34.679991 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ba81a-798c-432e-9d44-6d2e71dcb6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:34.680309 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ba81a-798c-432e-9d44-6d2e71dcb6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:34.680596 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ba81a-798c-432e-9d44-6d2e71dcb6a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:34.690648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be4665-57cd-4943-9e2d-46853b1752bd None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:34.697706 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ba81a-798c-432e-9d44-6d2e71dcb6a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:34.707780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-437ba81a-798c-432e-9d44-6d2e71dcb6a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:34.707780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:34.707780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:34.707780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:34.707780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:34.707780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:34.707780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:34.707780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:34.707780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:34.707780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:34.707780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:34.707780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:34.707780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:34.707780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:34.707780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:34.707780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:34.709432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:34.709432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:34.709432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:34.709432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:34.709432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:34.709432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:34.709432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:34.709432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:34.709432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:34.709432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:34.709432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:34.709432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:34.709432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:34.709432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:34.709432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:34.709432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:34.710341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:34.710341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:34.710341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:34.710341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:34.710341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:34.710341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:34.710341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:34.710341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:34.710341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:34.710341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:34.710341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:34.710341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:34.710341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 317/638] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 12:19:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:34.711031 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293ab73-b339-4803-bcf6-3ec6003c7171 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:34.711581 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293ab73-b339-4803-bcf6-3ec6003c7171 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:34.711581 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293ab73-b339-4803-bcf6-3ec6003c7171 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:34.714155 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 318/639] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:34.714541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293ab73-b339-4803-bcf6-3ec6003c7171 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:34.731111 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754f9b3c-061e-4460-b14a-754543397ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:34.731487 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f9b3c-061e-4460-b14a-754543397ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:34.731776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f9b3c-061e-4460-b14a-754543397ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:34.732058 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f9b3c-061e-4460-b14a-754543397ef3 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:34.732835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754f9b3c-061e-4460-b14a-754543397ef3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:34.733144 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754f9b3c-061e-4460-b14a-754543397ef3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:34.734526 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754f9b3c-061e-4460-b14a-754543397ef3 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:34.746776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 322/640] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:34.747566 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754f9b3c-061e-4460-b14a-754543397ef3 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:34.840030 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:34.845693 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293ab73-b339-4803-bcf6-3ec6003c7171 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:34.856512 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754f9b3c-061e-4460-b14a-754543397ef3 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:34.875230 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:34.882200 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6cf3b2-aee9-4b40-b6d5-5cc201ea1031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:34.882874 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cf3b2-aee9-4b40-b6d5-5cc201ea1031 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:34.883165 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cf3b2-aee9-4b40-b6d5-5cc201ea1031 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:34.883445 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cf3b2-aee9-4b40-b6d5-5cc201ea1031 None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:34.884200 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6cf3b2-aee9-4b40-b6d5-5cc201ea1031 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:34.884515 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6cf3b2-aee9-4b40-b6d5-5cc201ea1031 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:34.885896 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6cf3b2-aee9-4b40-b6d5-5cc201ea1031 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:34.896591 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38170fb5-8e6c-4a1a-9471-a4181ee912a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:34.897487 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38170fb5-8e6c-4a1a-9471-a4181ee912a3 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:34.897871 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 319/641] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:34.898079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38170fb5-8e6c-4a1a-9471-a4181ee912a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:34.898498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38170fb5-8e6c-4a1a-9471-a4181ee912a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:34.899117 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94981940-81b1-468e-b5c0-3dd422af751a None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:34.899409 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94981940-81b1-468e-b5c0-3dd422af751a None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:34.899693 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94981940-81b1-468e-b5c0-3dd422af751a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:34.899789 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38170fb5-8e6c-4a1a-9471-a4181ee912a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:34.900228 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38170fb5-8e6c-4a1a-9471-a4181ee912a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:34.901753 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38170fb5-8e6c-4a1a-9471-a4181ee912a3 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:34.915296 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 323/642] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 12:19:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:34.916182 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38170fb5-8e6c-4a1a-9471-a4181ee912a3 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:34.918463 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-94981940-81b1-468e-b5c0-3dd422af751a None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:34.919787 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-94981940-81b1-468e-b5c0-3dd422af751a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:34.990843 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 320/643] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:34.991997 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94981940-81b1-468e-b5c0-3dd422af751a None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:35.043208 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38170fb5-8e6c-4a1a-9471-a4181ee912a3 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:35.078851 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddde840-9195-445f-b677-6c91a3e30a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:35.079746 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddde840-9195-445f-b677-6c91a3e30a19 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:35.080031 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddde840-9195-445f-b677-6c91a3e30a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:35.080333 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddde840-9195-445f-b677-6c91a3e30a19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:35.087556 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddde840-9195-445f-b677-6c91a3e30a19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:35.089028 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddde840-9195-445f-b677-6c91a3e30a19 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:35.096142 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:35.104232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-0ddde840-9195-445f-b677-6c91a3e30a19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:35.104232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:35.104232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:35.104232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:35.104232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:35.104232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:35.104232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:35.104232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:35.104232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:35.104232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:35.104232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:35.104232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:35.104232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:19:35.104232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:19:35.104232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:19:35.105434 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:19:35.105434 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:35.105434 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:35.105434 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:35.105434 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:35.105434 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:35.105434 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:35.105434 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:35.105434 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:35.105434 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:35.105434 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:35.105434 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:35.105434 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:35.105434 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:35.105434 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:35.105434 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:35.105434 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:35.106417 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:19:35.106417 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:19:35.106417 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:19:35.106417 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:19:35.106417 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:19:35.106417 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:19:35.106417 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:35.106417 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:35.106417 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 324/644] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:35.106922 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddde840-9195-445f-b677-6c91a3e30a19 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:35.115887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94981940-81b1-468e-b5c0-3dd422af751a None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:35.150032 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:35.155715 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa553dc2-9f94-48e5-9c1b-941b53919ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:35.156857 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa553dc2-9f94-48e5-9c1b-941b53919ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:35.157193 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa553dc2-9f94-48e5-9c1b-941b53919ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:35.157487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa553dc2-9f94-48e5-9c1b-941b53919ed2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:35.165090 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa553dc2-9f94-48e5-9c1b-941b53919ed2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:35.166556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa553dc2-9f94-48e5-9c1b-941b53919ed2 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:35.180997 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-fa553dc2-9f94-48e5-9c1b-941b53919ed2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 12:19:35.180997 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:35.180997 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:35.180997 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:35.180997 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:35.180997 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:35.180997 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:35.180997 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:35.180997 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:35.180997 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:35.180997 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:35.180997 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:35.180997 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:19:35.180997 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:19:35.180997 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:19:35.180997 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:19:35.181976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:35.181976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:35.181976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:19:35.181976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:35.181976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:19:35.181976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:35.181976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:35.181976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:35.181976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:35.181976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:35.181976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:35.181976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:35.181976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:35.181976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:35.181976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:35.181976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:35.181976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:35.181976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:35.183072 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:35.183072 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:35.183072 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:19:35.183072 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:19:35.183072 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:19:35.183072 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:19:35.183072 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:19:35.183072 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:19:35.183072 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 12:19:35.183072 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:35.183072 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 321/645] 10.222.0.16 () {56 vars in 1093 bytes} [Mon May 23 12:19:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:35.183072 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa553dc2-9f94-48e5-9c1b-941b53919ed2 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:35.221311 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddde840-9195-445f-b677-6c91a3e30a19 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:35.261648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec62fe-1ba5-4aea-a58c-9780d9db2dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:35.262531 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec62fe-1ba5-4aea-a58c-9780d9db2dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:35.262883 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec62fe-1ba5-4aea-a58c-9780d9db2dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:35.263225 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec62fe-1ba5-4aea-a58c-9780d9db2dc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:35.275082 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec62fe-1ba5-4aea-a58c-9780d9db2dc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:35.276649 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec62fe-1ba5-4aea-a58c-9780d9db2dc2 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:35.293326 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa553dc2-9f94-48e5-9c1b-941b53919ed2 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:35.294675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-03ec62fe-1ba5-4aea-a58c-9780d9db2dc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:35.294675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:35.294675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:35.294675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:35.294675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:35.294675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:35.294675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:35.294675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:35.294675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:35.294675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:35.294675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:35.294675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:35.294675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:35.294675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:35.294675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:35.295705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:35.295705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:35.295705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:35.295705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:35.295705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:35.295705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:35.295705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:35.295705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:35.295705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:35.295705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:35.295705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:35.295705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:35.295705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:35.295705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:35.295705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:35.295705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:35.295705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:35.296920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:35.296920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:35.296920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:35.296920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:35.296920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 325/646] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:35.297197 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec62fe-1ba5-4aea-a58c-9780d9db2dc2 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:35.335180 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54127d4f-ef99-4420-84e5-476df1da666b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:35.335989 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54127d4f-ef99-4420-84e5-476df1da666b None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:35.336303 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54127d4f-ef99-4420-84e5-476df1da666b None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:35.336589 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54127d4f-ef99-4420-84e5-476df1da666b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:35.347035 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54127d4f-ef99-4420-84e5-476df1da666b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:35.354773 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-54127d4f-ef99-4420-84e5-476df1da666b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:35.354773 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:35.354773 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:35.354773 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:35.354773 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:35.354773 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:35.354773 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:35.354773 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:35.354773 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:35.354773 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:35.354773 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:35.354773 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:35.354773 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:35.354773 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:35.354773 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:35.354773 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:35.355719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:35.355719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:35.355719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:35.355719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:35.355719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:35.355719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:35.355719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:35.355719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:35.355719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:35.355719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:35.355719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:35.355719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:35.355719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:35.355719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:35.355719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:35.355719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:35.356614 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:35.356614 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:35.356614 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:35.356614 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:35.356614 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:35.356614 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:35.356614 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:35.356614 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:35.356614 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:35.356614 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:35.356614 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:35.356614 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:35.360739 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 322/647] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 12:19:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:35.361379 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54127d4f-ef99-4420-84e5-476df1da666b None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:35.372336 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:35.397173 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:35.408420 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec62fe-1ba5-4aea-a58c-9780d9db2dc2 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:35.441754 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e1ed3-f8ee-4722-b46d-5cce93c4e9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:35.443274 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e1ed3-f8ee-4722-b46d-5cce93c4e9a4 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:35.443824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e1ed3-f8ee-4722-b46d-5cce93c4e9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:35.444202 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e1ed3-f8ee-4722-b46d-5cce93c4e9a4 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:35.445056 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743e1ed3-f8ee-4722-b46d-5cce93c4e9a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:35.445439 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743e1ed3-f8ee-4722-b46d-5cce93c4e9a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:35.446935 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743e1ed3-f8ee-4722-b46d-5cce93c4e9a4 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:35.459720 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 326/648] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:19:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:35.460638 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e1ed3-f8ee-4722-b46d-5cce93c4e9a4 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:35.471825 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54127d4f-ef99-4420-84e5-476df1da666b None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:35.509799 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa47bea2-31b1-4ec5-95e4-3bd46f0dd631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:35.510472 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47bea2-31b1-4ec5-95e4-3bd46f0dd631 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:35.510760 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47bea2-31b1-4ec5-95e4-3bd46f0dd631 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:35.511046 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa47bea2-31b1-4ec5-95e4-3bd46f0dd631 None admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:35.511878 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa47bea2-31b1-4ec5-95e4-3bd46f0dd631 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:35.512207 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa47bea2-31b1-4ec5-95e4-3bd46f0dd631 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:35.513653 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa47bea2-31b1-4ec5-95e4-3bd46f0dd631 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:35.535002 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 323/649] 10.222.0.16 () {56 vars in 1133 bytes} [Mon May 23 12:19:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:35.535002 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa47bea2-31b1-4ec5-95e4-3bd46f0dd631 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:35.568807 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e1ed3-f8ee-4722-b46d-5cce93c4e9a4 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:35.601980 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:35.605620 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06128e9-ac28-4911-a279-3f823a9bbcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:35.606350 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06128e9-ac28-4911-a279-3f823a9bbcb1 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:35.606644 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06128e9-ac28-4911-a279-3f823a9bbcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:35.606922 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06128e9-ac28-4911-a279-3f823a9bbcb1 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:35.607650 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06128e9-ac28-4911-a279-3f823a9bbcb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:35.608018 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06128e9-ac28-4911-a279-3f823a9bbcb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:35.609493 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06128e9-ac28-4911-a279-3f823a9bbcb1 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:35.628021 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 327/650] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:35.628021 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06128e9-ac28-4911-a279-3f823a9bbcb1 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:35.638222 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:35.649482 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa47bea2-31b1-4ec5-95e4-3bd46f0dd631 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:35.686486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc1d3c2-dd7c-4c63-8c32-28464513b6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:35.687429 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1d3c2-dd7c-4c63-8c32-28464513b6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:35.687704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1d3c2-dd7c-4c63-8c32-28464513b6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:35.688078 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc1d3c2-dd7c-4c63-8c32-28464513b6e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:35.688955 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc1d3c2-dd7c-4c63-8c32-28464513b6e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:35.689354 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc1d3c2-dd7c-4c63-8c32-28464513b6e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:35.690825 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc1d3c2-dd7c-4c63-8c32-28464513b6e9 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:35.704651 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 324/651] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 12:19:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:35.704651 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc1d3c2-dd7c-4c63-8c32-28464513b6e9 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:35.743758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06128e9-ac28-4911-a279-3f823a9bbcb1 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:35.784705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5da26-9548-44dc-a1fb-85827eab665a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:35.785359 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5da26-9548-44dc-a1fb-85827eab665a None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:35.785643 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5da26-9548-44dc-a1fb-85827eab665a None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:35.785918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5da26-9548-44dc-a1fb-85827eab665a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:35.786894 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd5da26-9548-44dc-a1fb-85827eab665a None admin] RBAC: Adding query filter params (role.id=81d82471e2be4f0aa8164b75c16a9718, scope.project.id=839585501f414e1ab02501422510e54f, user.id=2483a351f5e943ad8c872a98a4717b0b) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:35.787468 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd5da26-9548-44dc-a1fb-85827eab665a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:35.789007 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd5da26-9548-44dc-a1fb-85827eab665a None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:35.804196 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 328/652] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:19:35 2022] GET /identity/v3/role_assignments?user.id=2483a351f5e943ad8c872a98a4717b0b&scope.project.id=839585501f414e1ab02501422510e54f&role.id=81d82471e2be4f0aa8164b75c16a9718 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:35.804196 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5da26-9548-44dc-a1fb-85827eab665a None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:35.850628 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc1d3c2-dd7c-4c63-8c32-28464513b6e9 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:35.886571 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:35.893448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:35.905371 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f8d329-ff55-45d5-ad5e-6a2261bad840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:35.906570 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8d329-ff55-45d5-ad5e-6a2261bad840 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:35.906863 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8d329-ff55-45d5-ad5e-6a2261bad840 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:35.907144 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f8d329-ff55-45d5-ad5e-6a2261bad840 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:35.918160 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5da26-9548-44dc-a1fb-85827eab665a None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:35.958569 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e38629-777e-446f-8f97-9cfe66be629b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:35.959880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e38629-777e-446f-8f97-9cfe66be629b None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:35.960297 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e38629-777e-446f-8f97-9cfe66be629b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:35.960677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e38629-777e-446f-8f97-9cfe66be629b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:35.969521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e38629-777e-446f-8f97-9cfe66be629b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:35.971434 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e38629-777e-446f-8f97-9cfe66be629b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:35.988014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-46e38629-777e-446f-8f97-9cfe66be629b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 12:19:35.988014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:35.988014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:35.988014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:35.988014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:35.988014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:35.988014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:35.988014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:35.988014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:35.988014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:35.988014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:35.988014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:35.988014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:19:35.988014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:19:35.988014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:19:35.988014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:19:35.989771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:35.989771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:35.989771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:19:35.989771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:35.989771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:19:35.989771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:35.989771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:35.989771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:35.989771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:35.989771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:35.989771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:35.989771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:35.989771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:35.989771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:35.989771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:35.989771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:35.989771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:35.989771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:35.991901 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:35.991901 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:35.991901 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:19:35.991901 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:19:35.991901 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:19:35.991901 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:19:35.991901 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:19:35.991901 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:19:35.991901 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 12:19:35.991901 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:35.991901 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 329/653] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:19:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:35.997428 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e38629-777e-446f-8f97-9cfe66be629b None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:36.005040 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f8d329-ff55-45d5-ad5e-6a2261bad840 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:36.007394 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f8d329-ff55-45d5-ad5e-6a2261bad840 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:36.119985 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e38629-777e-446f-8f97-9cfe66be629b None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:36.129922 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 325/654] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:19:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:19:36.131549 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f8d329-ff55-45d5-ad5e-6a2261bad840 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:36.163653 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357b316-5c8d-4185-b55f-5696385bd70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:36.164409 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357b316-5c8d-4185-b55f-5696385bd70d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:36.164700 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357b316-5c8d-4185-b55f-5696385bd70d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:36.164981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357b316-5c8d-4185-b55f-5696385bd70d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:36.169361 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:36.173765 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:36.179326 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357b316-5c8d-4185-b55f-5696385bd70d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:36.181343 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357b316-5c8d-4185-b55f-5696385bd70d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:36.206850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-5357b316-5c8d-4185-b55f-5696385bd70d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:36.206850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:36.206850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:36.206850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:36.206850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:36.206850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:36.206850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:36.206850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:36.206850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:36.206850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:36.206850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:36.206850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:36.206850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:36.206850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:36.206850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:36.207866 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:36.207866 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:36.207866 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:36.207866 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:36.207866 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:36.207866 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:36.207866 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:36.207866 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:36.207866 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:36.207866 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:36.207866 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:36.207866 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:36.207866 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:36.207866 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:36.207866 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:36.207866 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:36.207866 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:36.209357 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:36.209357 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:36.209357 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:36.209357 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:36.212260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 330/655] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:36.214177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357b316-5c8d-4185-b55f-5696385bd70d None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:36.248968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f8d329-ff55-45d5-ad5e-6a2261bad840 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:36.288222 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45193392-2ca2-49b7-90d7-f99e75ba3c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:36.290207 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45193392-2ca2-49b7-90d7-f99e75ba3c6c None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:36.290207 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45193392-2ca2-49b7-90d7-f99e75ba3c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:36.290207 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45193392-2ca2-49b7-90d7-f99e75ba3c6c None admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:36.290502 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45193392-2ca2-49b7-90d7-f99e75ba3c6c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:36.290585 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45193392-2ca2-49b7-90d7-f99e75ba3c6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:36.292109 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45193392-2ca2-49b7-90d7-f99e75ba3c6c None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:36.318304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 326/656] 10.222.0.16 () {56 vars in 1137 bytes} [Mon May 23 12:19:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:36.319324 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45193392-2ca2-49b7-90d7-f99e75ba3c6c None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:36.369800 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357b316-5c8d-4185-b55f-5696385bd70d None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:36.415389 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:36.418688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0491b4b-5338-4566-ab37-df8fe7bc6f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:36.419801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0491b4b-5338-4566-ab37-df8fe7bc6f11 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:36.420426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0491b4b-5338-4566-ab37-df8fe7bc6f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:36.420945 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0491b4b-5338-4566-ab37-df8fe7bc6f11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:36.433381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45193392-2ca2-49b7-90d7-f99e75ba3c6c None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:36.437513 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0491b4b-5338-4566-ab37-df8fe7bc6f11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:36.438965 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0491b4b-5338-4566-ab37-df8fe7bc6f11 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:36.457625 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-c0491b4b-5338-4566-ab37-df8fe7bc6f11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:36.457625 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:36.457625 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:36.457625 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:36.457625 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:36.457625 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:36.457625 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:36.457625 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:36.457625 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:36.457625 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:36.457625 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:36.457625 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:36.457625 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:36.457625 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:36.457625 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:36.459060 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:36.459060 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:36.459060 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:36.459060 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:36.459060 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:36.459060 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:36.459060 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:36.459060 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:36.459060 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:36.459060 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:36.459060 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:36.459060 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:36.459060 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:36.459060 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:36.459060 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:36.459060 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:36.459060 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:36.460116 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:36.460116 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:36.460116 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:36.460116 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:36.460116 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 331/657] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:36.460116 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0491b4b-5338-4566-ab37-df8fe7bc6f11 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:36.479936 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d759551-3569-4a12-b9e8-65af4d5f038f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:36.481455 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d759551-3569-4a12-b9e8-65af4d5f038f None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:36.481885 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d759551-3569-4a12-b9e8-65af4d5f038f None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:36.482305 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d759551-3569-4a12-b9e8-65af4d5f038f None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:36.483172 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d759551-3569-4a12-b9e8-65af4d5f038f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:36.484041 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d759551-3569-4a12-b9e8-65af4d5f038f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:36.486016 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d759551-3569-4a12-b9e8-65af4d5f038f None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:36.502581 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 327/658] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:36.504130 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d759551-3569-4a12-b9e8-65af4d5f038f None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:36.507722 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:36.603152 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0491b4b-5338-4566-ab37-df8fe7bc6f11 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:36.627329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d759551-3569-4a12-b9e8-65af4d5f038f None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:36.651305 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe0129-b222-4450-a54c-a92ee5fd04c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:36.653369 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe0129-b222-4450-a54c-a92ee5fd04c3 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:36.653885 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe0129-b222-4450-a54c-a92ee5fd04c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:36.654638 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe0129-b222-4450-a54c-a92ee5fd04c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:36.663667 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03917fa6-7ee8-4e46-8415-e9ec662372c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:36.664356 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03917fa6-7ee8-4e46-8415-e9ec662372c6 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:36.664817 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03917fa6-7ee8-4e46-8415-e9ec662372c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:36.665104 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03917fa6-7ee8-4e46-8415-e9ec662372c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:36.665891 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03917fa6-7ee8-4e46-8415-e9ec662372c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:36.666140 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03917fa6-7ee8-4e46-8415-e9ec662372c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:36.667778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03917fa6-7ee8-4e46-8415-e9ec662372c6 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:36.682285 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 328/659] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:36.686973 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03917fa6-7ee8-4e46-8415-e9ec662372c6 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:36.699524 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:36.743569 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffe0129-b222-4450-a54c-a92ee5fd04c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:36.745051 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffe0129-b222-4450-a54c-a92ee5fd04c3 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:36.788638 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:36.810415 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03917fa6-7ee8-4e46-8415-e9ec662372c6 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:36.847727 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91e63b-8515-46c3-9fa9-0ecf1518fb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:36.849494 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91e63b-8515-46c3-9fa9-0ecf1518fb6a None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:36.849910 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91e63b-8515-46c3-9fa9-0ecf1518fb6a None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:36.850258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91e63b-8515-46c3-9fa9-0ecf1518fb6a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:36.862290 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 332/660] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:19:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:19:36.862448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd91e63b-8515-46c3-9fa9-0ecf1518fb6a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:36.862566 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe0129-b222-4450-a54c-a92ee5fd04c3 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:36.862655 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd91e63b-8515-46c3-9fa9-0ecf1518fb6a None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:36.878733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-dd91e63b-8515-46c3-9fa9-0ecf1518fb6a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 23 12:19:36.878733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:36.878733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:36.878733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:36.878733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:36.878733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:36.878733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:36.878733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:36.878733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:36.878733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:36.878733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:36.878733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:36.878733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:19:36.878733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:19:36.878733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:19:36.878733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:19:36.879881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:36.879881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:36.879881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:19:36.879881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:36.879881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:19:36.879881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:36.879881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:36.879881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:36.879881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:36.879881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:36.879881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:36.879881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:36.879881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:36.879881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:36.879881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:36.879881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:36.879881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:36.879881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:36.881191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:36.881191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:36.881191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:19:36.881191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:19:36.881191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:19:36.881191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:19:36.881191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:19:36.881191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:19:36.881191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 23 12:19:36.881191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:36.884659 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 329/661] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 12:19:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:36.885797 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e079a9e-60b2-4370-b33e-f5be58db3ef6 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:36.886096 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e079a9e-60b2-4370-b33e-f5be58db3ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:36.886379 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e079a9e-60b2-4370-b33e-f5be58db3ef6 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:36.891600 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 330/662] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:36.891600 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e079a9e-60b2-4370-b33e-f5be58db3ef6 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:36.959130 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:36.972703 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe0129-b222-4450-a54c-a92ee5fd04c3 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:36.998553 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e079a9e-60b2-4370-b33e-f5be58db3ef6 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:37.011246 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b62c9-9502-4c0e-904a-59dcf804f6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:37.012155 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b62c9-9502-4c0e-904a-59dcf804f6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:37.012570 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b62c9-9502-4c0e-904a-59dcf804f6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:37.012952 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b62c9-9502-4c0e-904a-59dcf804f6a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:37.020012 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b62c9-9502-4c0e-904a-59dcf804f6a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:37.022211 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b62c9-9502-4c0e-904a-59dcf804f6a4 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:37.027471 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:37.040349 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6658d-7f1e-48f0-bed4-d9f13b76ee4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:37.040880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-d78b62c9-9502-4c0e-904a-59dcf804f6a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:37.040880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:37.040880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:37.040880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:37.040880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:37.040880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:37.040880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:37.040880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:37.040880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:37.040880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:37.040880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:37.040880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:37.040880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:19:37.040880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:19:37.040880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:19:37.041831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:19:37.041831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:37.041831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:37.041831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:37.041831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:37.041831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:37.041831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:37.041831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:37.041831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:37.041831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:37.041831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:37.041831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:37.041831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:37.041831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:37.041831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:37.041831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:37.041831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:37.042777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:19:37.042777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:19:37.042777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:19:37.042777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:19:37.042777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:19:37.042777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:19:37.042777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:37.042777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:37.043212 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6658d-7f1e-48f0-bed4-d9f13b76ee4e None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:37.043212 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6658d-7f1e-48f0-bed4-d9f13b76ee4e None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:37.043212 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6658d-7f1e-48f0-bed4-d9f13b76ee4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:37.045585 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 333/663] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:37.050674 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b62c9-9502-4c0e-904a-59dcf804f6a4 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:37.054465 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6658d-7f1e-48f0-bed4-d9f13b76ee4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:37.056784 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6658d-7f1e-48f0-bed4-d9f13b76ee4e None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:37.077444 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-53f6658d-7f1e-48f0-bed4-d9f13b76ee4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:37.077444 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:37.077444 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:37.077444 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:37.077444 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:37.077444 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:37.077444 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:37.077444 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:37.077444 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:37.077444 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:37.077444 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:37.077444 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:37.077444 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:37.077444 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:37.077444 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:37.078420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:37.078420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:37.078420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:37.078420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:37.078420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:37.078420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:37.078420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:37.078420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:37.078420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:37.078420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:37.078420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:37.078420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:37.078420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:37.078420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:37.078420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:37.078420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:37.078420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:37.079329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:37.079329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:37.079329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:37.079329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:37.079329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 331/664] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:37.080145 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04c8ac-88a1-477a-b0c5-0dd8f569f1d9 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:37.080145 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04c8ac-88a1-477a-b0c5-0dd8f569f1d9 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:37.080454 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04c8ac-88a1-477a-b0c5-0dd8f569f1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:37.097810 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-de04c8ac-88a1-477a-b0c5-0dd8f569f1d9 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:37.099677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-de04c8ac-88a1-477a-b0c5-0dd8f569f1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:37.161327 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b62c9-9502-4c0e-904a-59dcf804f6a4 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:37.184225 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 332/665] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:37.184225 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04c8ac-88a1-477a-b0c5-0dd8f569f1d9 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:37.209880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6037c25-cbba-4ab8-a9c0-bc2062ad9aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:37.210485 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6037c25-cbba-4ab8-a9c0-bc2062ad9aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:37.210776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6037c25-cbba-4ab8-a9c0-bc2062ad9aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:37.211067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6037c25-cbba-4ab8-a9c0-bc2062ad9aa3 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:37.211908 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6037c25-cbba-4ab8-a9c0-bc2062ad9aa3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:37.212255 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6037c25-cbba-4ab8-a9c0-bc2062ad9aa3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:37.213685 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6037c25-cbba-4ab8-a9c0-bc2062ad9aa3 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:37.231538 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 334/666] 10.222.0.16 () {56 vars in 1139 bytes} [Mon May 23 12:19:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:37.233165 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6037c25-cbba-4ab8-a9c0-bc2062ad9aa3 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:37.237501 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:37.292641 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04c8ac-88a1-477a-b0c5-0dd8f569f1d9 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:37.299706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:37.329859 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a23e3-9992-4ec0-a6f4-22ed24541deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:37.330616 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a23e3-9992-4ec0-a6f4-22ed24541deb None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:37.330973 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a23e3-9992-4ec0-a6f4-22ed24541deb None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:37.331325 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a23e3-9992-4ec0-a6f4-22ed24541deb None admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:37.332205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a23e3-9992-4ec0-a6f4-22ed24541deb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:37.332611 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a23e3-9992-4ec0-a6f4-22ed24541deb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:37.334405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a23e3-9992-4ec0-a6f4-22ed24541deb None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:37.346217 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 333/667] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:19:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:37.347010 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a23e3-9992-4ec0-a6f4-22ed24541deb None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:37.354147 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6037c25-cbba-4ab8-a9c0-bc2062ad9aa3 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:37.393177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5764ef8-d0fe-4fc2-9a1e-7abab22b007b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:37.393867 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5764ef8-d0fe-4fc2-9a1e-7abab22b007b None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:37.394156 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5764ef8-d0fe-4fc2-9a1e-7abab22b007b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:37.394434 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5764ef8-d0fe-4fc2-9a1e-7abab22b007b None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:37.395162 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5764ef8-d0fe-4fc2-9a1e-7abab22b007b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:37.395469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5764ef8-d0fe-4fc2-9a1e-7abab22b007b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:37.396957 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5764ef8-d0fe-4fc2-9a1e-7abab22b007b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:37.408525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 335/668] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:37.409415 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5764ef8-d0fe-4fc2-9a1e-7abab22b007b None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:37.459767 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a23e3-9992-4ec0-a6f4-22ed24541deb None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:37.469251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:37.497108 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc194dd-1209-4d37-886b-ab175fbc3ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:37.497970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc194dd-1209-4d37-886b-ab175fbc3ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:37.498262 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc194dd-1209-4d37-886b-ab175fbc3ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:37.498544 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc194dd-1209-4d37-886b-ab175fbc3ac1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:37.509927 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc194dd-1209-4d37-886b-ab175fbc3ac1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:37.517606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-7dc194dd-1209-4d37-886b-ab175fbc3ac1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:37.517606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:37.517606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:37.517606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:37.517606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:37.517606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:37.517606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:37.517606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:37.517606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:37.517606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:37.517606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:37.517606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:37.517606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:37.517606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:37.517606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:37.517606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:37.518544 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:37.518544 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:37.518544 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:37.518544 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:37.518544 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:37.518544 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:37.518544 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:37.518544 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:37.518544 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:37.518544 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:37.518544 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:37.518544 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:37.518544 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:37.518544 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:37.518544 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:37.518544 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:37.519415 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:37.519415 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:37.519415 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:37.519415 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:37.519415 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:37.519415 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:37.519415 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:37.519415 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:37.519415 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:37.519415 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:37.519415 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:37.519415 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:37.519415 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 334/669] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 12:19:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:37.520134 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc194dd-1209-4d37-886b-ab175fbc3ac1 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:37.535743 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5764ef8-d0fe-4fc2-9a1e-7abab22b007b None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:37.555724 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:37.572969 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1fd88f-96ab-4c7c-a594-93acf6826f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:37.573611 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fd88f-96ab-4c7c-a594-93acf6826f3e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:37.573989 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fd88f-96ab-4c7c-a594-93acf6826f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:37.574270 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1fd88f-96ab-4c7c-a594-93acf6826f3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:37.633708 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc194dd-1209-4d37-886b-ab175fbc3ac1 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:37.646679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1fd88f-96ab-4c7c-a594-93acf6826f3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:37.648122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1fd88f-96ab-4c7c-a594-93acf6826f3e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:37.670041 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf6f9b0-91fe-43f7-a045-92404b195f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:37.670839 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6f9b0-91fe-43f7-a045-92404b195f65 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:37.671240 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6f9b0-91fe-43f7-a045-92404b195f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:37.671625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6f9b0-91fe-43f7-a045-92404b195f65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:37.672589 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf6f9b0-91fe-43f7-a045-92404b195f65 None admin] RBAC: Adding query filter params (role.id=81d82471e2be4f0aa8164b75c16a9718, scope.project.id=839585501f414e1ab02501422510e54f, user.id=b83535b443a84275a160cd84cb3ad5a0) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:37.673015 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf6f9b0-91fe-43f7-a045-92404b195f65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:37.674577 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf6f9b0-91fe-43f7-a045-92404b195f65 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:37.692202 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 335/670] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:19:37 2022] GET /identity/v3/role_assignments?user.id=b83535b443a84275a160cd84cb3ad5a0&scope.project.id=839585501f414e1ab02501422510e54f&role.id=81d82471e2be4f0aa8164b75c16a9718 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:37.692202 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf6f9b0-91fe-43f7-a045-92404b195f65 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:37.729462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:37.792314 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 336/671] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:19:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:19:37.793995 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1fd88f-96ab-4c7c-a594-93acf6826f3e None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:37.815528 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf6f9b0-91fe-43f7-a045-92404b195f65 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:37.816312 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:37.856536 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a29f1-553e-45a4-aad7-b01e238e0120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:37.857778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a29f1-553e-45a4-aad7-b01e238e0120 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:37.858294 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a29f1-553e-45a4-aad7-b01e238e0120 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:37.858779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a29f1-553e-45a4-aad7-b01e238e0120 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:37.869698 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a29f1-553e-45a4-aad7-b01e238e0120 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:37.872301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a29f1-553e-45a4-aad7-b01e238e0120 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:37.889075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-8f2a29f1-553e-45a4-aad7-b01e238e0120 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:37.889075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:37.889075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:37.889075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:37.889075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:37.889075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:37.889075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:37.889075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:37.889075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:37.889075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:37.889075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:37.889075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:37.889075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:19:37.889075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:19:37.889075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:19:37.890267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:19:37.890267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:37.890267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:37.890267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:37.890267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:37.890267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:37.890267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:37.890267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:37.890267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:37.890267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:37.890267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:37.890267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:37.890267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:37.890267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:37.890267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:37.890267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:37.890267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:37.891302 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:19:37.891302 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:19:37.891302 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:19:37.891302 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:19:37.891302 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:19:37.891302 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:19:37.891302 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:37.891302 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:37.893729 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 336/672] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:37.895482 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a29f1-553e-45a4-aad7-b01e238e0120 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:37.936855 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1fd88f-96ab-4c7c-a594-93acf6826f3e None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:37.980981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb1ed1-e737-456e-ae7b-20bfe23fb7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:37.981695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb1ed1-e737-456e-ae7b-20bfe23fb7aa None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:37.981864 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb1ed1-e737-456e-ae7b-20bfe23fb7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:37.982143 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb1ed1-e737-456e-ae7b-20bfe23fb7aa None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:37.982895 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb1ed1-e737-456e-ae7b-20bfe23fb7aa None admin] RBAC: Adding query filter params (name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:37.983205 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb1ed1-e737-456e-ae7b-20bfe23fb7aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:37.984623 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb1ed1-e737-456e-ae7b-20bfe23fb7aa None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:37.994804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 337/673] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 12:19:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:37.995551 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb1ed1-e737-456e-ae7b-20bfe23fb7aa None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:38.016573 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:38.030229 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a29f1-553e-45a4-aad7-b01e238e0120 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:38.071164 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c6ad0-73af-4439-98b2-7c7eaea81801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:38.072003 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c6ad0-73af-4439-98b2-7c7eaea81801 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:38.072329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c6ad0-73af-4439-98b2-7c7eaea81801 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:38.072610 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c6ad0-73af-4439-98b2-7c7eaea81801 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:38.085694 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269c6ad0-73af-4439-98b2-7c7eaea81801 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:38.087093 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269c6ad0-73af-4439-98b2-7c7eaea81801 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:38.107673 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:38.109411 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-269c6ad0-73af-4439-98b2-7c7eaea81801 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:38.109411 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:38.109411 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:38.109411 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:38.109411 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:38.109411 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:38.109411 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:38.109411 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:38.109411 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:38.109411 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:38.109411 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:38.109411 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:38.109411 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:38.109411 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:38.109411 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:38.111160 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:38.111160 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:38.111160 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:38.111160 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:38.111160 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:38.111160 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:38.111160 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:38.111160 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:38.111160 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:38.111160 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:38.111160 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:38.111160 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:38.111160 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:38.111160 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:38.111160 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:38.111160 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:38.111160 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:38.112093 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:38.112093 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:38.112093 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:38.112093 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:38.112093 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 337/674] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:38.112093 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c6ad0-73af-4439-98b2-7c7eaea81801 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:38.142091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb1ed1-e737-456e-ae7b-20bfe23fb7aa None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:38.189399 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5faacf-3595-4129-9c5c-2512d15c2e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:38.190695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5faacf-3595-4129-9c5c-2512d15c2e61 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:38.192349 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5faacf-3595-4129-9c5c-2512d15c2e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:38.192885 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5faacf-3595-4129-9c5c-2512d15c2e61 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:38.194052 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5faacf-3595-4129-9c5c-2512d15c2e61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:38.194574 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5faacf-3595-4129-9c5c-2512d15c2e61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:38.196268 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5faacf-3595-4129-9c5c-2512d15c2e61 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:38.213755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 338/675] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:19:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:38.215239 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5faacf-3595-4129-9c5c-2512d15c2e61 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:38.227994 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c6ad0-73af-4439-98b2-7c7eaea81801 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:38.276299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e5f87-4a96-4cce-8778-24066c49e7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:38.277337 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e5f87-4a96-4cce-8778-24066c49e7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:38.277337 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e5f87-4a96-4cce-8778-24066c49e7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:38.277570 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e5f87-4a96-4cce-8778-24066c49e7c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:38.288720 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e5f87-4a96-4cce-8778-24066c49e7c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:38.290237 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07e5f87-4a96-4cce-8778-24066c49e7c5 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:38.293056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:38.311372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-d07e5f87-4a96-4cce-8778-24066c49e7c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:38.311372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:38.311372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:38.311372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:38.311372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:38.311372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:38.311372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:38.311372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:38.311372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:38.311372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:38.311372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:38.311372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:38.311372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:38.311372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:38.311372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:38.312348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:38.312348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:38.312348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:38.312348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:38.312348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:38.312348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:38.312348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:38.312348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:38.312348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:38.312348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:38.312348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:38.312348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:38.312348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:38.312348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:38.312348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:38.312348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:38.312348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:38.313263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:38.313263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:38.313263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:38.313263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:38.313263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 338/676] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:38.314017 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e5f87-4a96-4cce-8778-24066c49e7c5 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:38.341822 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5faacf-3595-4129-9c5c-2512d15c2e61 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:38.378609 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da51b5-4586-46de-b02d-2a2e5f020f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:38.379524 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da51b5-4586-46de-b02d-2a2e5f020f80 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:38.379817 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da51b5-4586-46de-b02d-2a2e5f020f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:38.380105 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da51b5-4586-46de-b02d-2a2e5f020f80 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:38.382726 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8da51b5-4586-46de-b02d-2a2e5f020f80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:38.383043 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8da51b5-4586-46de-b02d-2a2e5f020f80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:38.384444 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8da51b5-4586-46de-b02d-2a2e5f020f80 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:38.391018 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:38.401117 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 339/677] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:38.401117 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a6dfd-e673-4c01-a468-69b58398b131 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:38.401117 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a6dfd-e673-4c01-a468-69b58398b131 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:38.401117 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8a6dfd-e673-4c01-a468-69b58398b131 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:38.401117 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 340/678] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:38.401117 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a6dfd-e673-4c01-a468-69b58398b131 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:38.429445 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e5f87-4a96-4cce-8778-24066c49e7c5 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:38.465127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f551f9-4aa9-4875-bdfd-4a0db2adf743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:38.465779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f551f9-4aa9-4875-bdfd-4a0db2adf743 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:38.466070 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f551f9-4aa9-4875-bdfd-4a0db2adf743 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:38.466345 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f551f9-4aa9-4875-bdfd-4a0db2adf743 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:38.467169 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f551f9-4aa9-4875-bdfd-4a0db2adf743 None admin] RBAC: Adding query filter params (role.id=81d82471e2be4f0aa8164b75c16a9718, scope.project.id=839585501f414e1ab02501422510e54f, user.id=e12ef3e115544a51a6bf9b08d265fefb) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:38.467480 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f551f9-4aa9-4875-bdfd-4a0db2adf743 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:38.468944 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f551f9-4aa9-4875-bdfd-4a0db2adf743 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:38.484199 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 339/679] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:19:38 2022] GET /identity/v3/role_assignments?user.id=e12ef3e115544a51a6bf9b08d265fefb&scope.project.id=839585501f414e1ab02501422510e54f&role.id=81d82471e2be4f0aa8164b75c16a9718 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:38.484199 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f551f9-4aa9-4875-bdfd-4a0db2adf743 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:38.529044 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8a6dfd-e673-4c01-a468-69b58398b131 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:38.551928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:38.583135 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7aa374-1f7c-4a8b-9356-5ea6961bf823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:38.584564 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7aa374-1f7c-4a8b-9356-5ea6961bf823 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:38.585204 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7aa374-1f7c-4a8b-9356-5ea6961bf823 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:38.585592 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7aa374-1f7c-4a8b-9356-5ea6961bf823 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:38.586891 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7aa374-1f7c-4a8b-9356-5ea6961bf823 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:38.587293 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7aa374-1f7c-4a8b-9356-5ea6961bf823 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:38.588798 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7aa374-1f7c-4a8b-9356-5ea6961bf823 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:38.603904 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f551f9-4aa9-4875-bdfd-4a0db2adf743 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:38.608340 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 341/680] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:38.609958 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e7369-c3ce-413c-9211-58c838c805ca None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:38.610317 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e7369-c3ce-413c-9211-58c838c805ca None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:38.610656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e7369-c3ce-413c-9211-58c838c805ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:38.636602 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-7e0e7369-c3ce-413c-9211-58c838c805ca None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:38.638313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-7e0e7369-c3ce-413c-9211-58c838c805ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:38.656840 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c2252-8de7-40b8-8a0e-5f41e78a09dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:38.658045 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c2252-8de7-40b8-8a0e-5f41e78a09dd None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:38.658286 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c2252-8de7-40b8-8a0e-5f41e78a09dd None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:38.658659 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c2252-8de7-40b8-8a0e-5f41e78a09dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:38.667929 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c2252-8de7-40b8-8a0e-5f41e78a09dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:38.669734 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c2252-8de7-40b8-8a0e-5f41e78a09dd None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:38.687442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-5e0c2252-8de7-40b8-8a0e-5f41e78a09dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:38.687442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:38.687442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:38.687442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:38.687442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:38.687442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:38.687442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:38.687442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:38.687442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:38.687442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:38.687442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:38.687442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:38.687442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:19:38.687442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:19:38.687442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:19:38.688646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:19:38.688646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:38.688646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:38.688646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:38.688646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:38.688646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:38.688646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:38.688646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:38.688646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:38.688646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:38.688646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:38.688646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:38.688646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:38.688646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:38.688646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:38.688646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:38.688646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:38.689710 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:19:38.689710 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:19:38.689710 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:19:38.689710 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:19:38.689710 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:19:38.689710 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:19:38.689710 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:38.689710 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:38.690970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 340/681] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:38.692118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c2252-8de7-40b8-8a0e-5f41e78a09dd None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:38.708329 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:38.725956 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 342/682] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:38.728607 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e7369-c3ce-413c-9211-58c838c805ca None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:38.825126 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c2252-8de7-40b8-8a0e-5f41e78a09dd None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:38.859582 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:38.871395 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e7369-c3ce-413c-9211-58c838c805ca None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:38.877994 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e4150-0dba-4852-8c01-b00d06c8eb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:38.879476 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e4150-0dba-4852-8c01-b00d06c8eb4b None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:38.880039 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e4150-0dba-4852-8c01-b00d06c8eb4b None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:38.880760 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e4150-0dba-4852-8c01-b00d06c8eb4b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:38.889640 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e4150-0dba-4852-8c01-b00d06c8eb4b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:38.891207 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e4150-0dba-4852-8c01-b00d06c8eb4b None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:38.907414 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-4a7e4150-0dba-4852-8c01-b00d06c8eb4b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 12:19:38.907414 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:38.907414 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:38.907414 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:38.907414 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:38.907414 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:38.907414 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:38.907414 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:38.907414 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:38.907414 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:38.907414 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:38.907414 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:38.907414 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:19:38.907414 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:19:38.907414 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:19:38.907414 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:19:38.908733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:38.908733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:38.908733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:19:38.908733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:38.908733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:19:38.908733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:38.908733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:38.908733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:38.908733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:38.908733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:38.908733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:38.908733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:38.908733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:38.908733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:38.908733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:38.908733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:38.908733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:38.908733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:38.909907 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:38.909907 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:38.909907 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:19:38.909907 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:19:38.909907 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:19:38.909907 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:19:38.909907 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:19:38.909907 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:19:38.909907 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 12:19:38.909907 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:38.912237 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 341/683] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:19:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:38.920196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e4150-0dba-4852-8c01-b00d06c8eb4b None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:38.924319 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ce6206-a87e-40be-a0e7-214927bb6ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:38.925002 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce6206-a87e-40be-a0e7-214927bb6ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:38.925288 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce6206-a87e-40be-a0e7-214927bb6ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:38.925567 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce6206-a87e-40be-a0e7-214927bb6ed9 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:38.926370 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ce6206-a87e-40be-a0e7-214927bb6ed9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:38.926682 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ce6206-a87e-40be-a0e7-214927bb6ed9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:38.928790 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ce6206-a87e-40be-a0e7-214927bb6ed9 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:38.942954 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 343/684] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:19:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:38.944005 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ce6206-a87e-40be-a0e7-214927bb6ed9 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:39.002886 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:39.042349 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e4150-0dba-4852-8c01-b00d06c8eb4b None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:39.063596 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ce6206-a87e-40be-a0e7-214927bb6ed9 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:39.078568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fef829-95b0-4d40-8c7a-8ba7521e53ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:39.079304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fef829-95b0-4d40-8c7a-8ba7521e53ef None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:39.079591 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fef829-95b0-4d40-8c7a-8ba7521e53ef None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:39.079869 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fef829-95b0-4d40-8c7a-8ba7521e53ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:39.090830 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fef829-95b0-4d40-8c7a-8ba7521e53ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:39.092324 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fef829-95b0-4d40-8c7a-8ba7521e53ef None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:39.104029 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368c30d-0885-46a4-bafa-9b33eda90ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:39.105031 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368c30d-0885-46a4-bafa-9b33eda90ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:39.105522 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368c30d-0885-46a4-bafa-9b33eda90ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:39.106001 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368c30d-0885-46a4-bafa-9b33eda90ed9 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:39.107080 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3368c30d-0885-46a4-bafa-9b33eda90ed9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:39.108544 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3368c30d-0885-46a4-bafa-9b33eda90ed9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:39.110126 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3368c30d-0885-46a4-bafa-9b33eda90ed9 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:39.111906 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-e9fef829-95b0-4d40-8c7a-8ba7521e53ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:39.111906 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:39.111906 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:39.111906 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:39.111906 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:39.111906 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:39.111906 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:39.111906 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:39.111906 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:39.111906 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:39.111906 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:39.111906 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:39.111906 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:39.111906 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:39.111906 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:39.112841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:39.112841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:39.112841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:39.112841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:39.112841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:39.112841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:39.112841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:39.112841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:39.112841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:39.112841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:39.112841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:39.112841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:39.112841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:39.112841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:39.112841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:39.112841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:39.112841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:39.113751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:39.113751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:39.113751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:39.113751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:39.113751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 342/685] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:39.114362 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fef829-95b0-4d40-8c7a-8ba7521e53ef None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:39.136091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 344/686] 10.222.0.16 () {56 vars in 1135 bytes} [Mon May 23 12:19:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:39.137084 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368c30d-0885-46a4-bafa-9b33eda90ed9 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:39.149480 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:39.226001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fef829-95b0-4d40-8c7a-8ba7521e53ef None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:39.263300 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b67d01-1acd-4dd5-94ab-7bf05d0a58c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:39.264014 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b67d01-1acd-4dd5-94ab-7bf05d0a58c5 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:39.264349 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b67d01-1acd-4dd5-94ab-7bf05d0a58c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:39.264763 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b67d01-1acd-4dd5-94ab-7bf05d0a58c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:39.265597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b67d01-1acd-4dd5-94ab-7bf05d0a58c5 None admin] RBAC: Adding query filter params (role.id=81d82471e2be4f0aa8164b75c16a9718, scope.project.id=839585501f414e1ab02501422510e54f, user.id=b6677be3ddf647e2b971bb3ba2c34a3a) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:39.265915 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b67d01-1acd-4dd5-94ab-7bf05d0a58c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:39.267350 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b67d01-1acd-4dd5-94ab-7bf05d0a58c5 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:39.275024 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368c30d-0885-46a4-bafa-9b33eda90ed9 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:39.280344 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 343/687] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:19:39 2022] GET /identity/v3/role_assignments?user.id=b6677be3ddf647e2b971bb3ba2c34a3a&scope.project.id=839585501f414e1ab02501422510e54f&role.id=81d82471e2be4f0aa8164b75c16a9718 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:39.281476 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b67d01-1acd-4dd5-94ab-7bf05d0a58c5 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:39.284348 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:39.321405 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7839a241-3953-45fd-a88e-6f3d7e545433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:39.322647 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839a241-3953-45fd-a88e-6f3d7e545433 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:39.323101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839a241-3953-45fd-a88e-6f3d7e545433 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:39.323556 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839a241-3953-45fd-a88e-6f3d7e545433 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:39.325161 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7839a241-3953-45fd-a88e-6f3d7e545433 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:39.325659 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7839a241-3953-45fd-a88e-6f3d7e545433 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:39.327432 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7839a241-3953-45fd-a88e-6f3d7e545433 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:39.343711 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 345/688] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:39.350709 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7839a241-3953-45fd-a88e-6f3d7e545433 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:39.477694 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7839a241-3953-45fd-a88e-6f3d7e545433 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:39.519369 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cbb7b3-4671-423a-b0a6-c7d968861fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:39.520457 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbb7b3-4671-423a-b0a6-c7d968861fd9 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:39.521991 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbb7b3-4671-423a-b0a6-c7d968861fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:39.522608 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cbb7b3-4671-423a-b0a6-c7d968861fd9 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:39.523515 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cbb7b3-4671-423a-b0a6-c7d968861fd9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:39.525185 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cbb7b3-4671-423a-b0a6-c7d968861fd9 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:39.542907 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:39.551595 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b67d01-1acd-4dd5-94ab-7bf05d0a58c5 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:39.558963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 346/689] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 12:19:39 2022] POST /identity/v3/users => generated 254 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:19:39.599702 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e39643a-4e2e-40b9-a5c3-28a1d9fe6220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:39.600407 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39643a-4e2e-40b9-a5c3-28a1d9fe6220 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:39.600721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39643a-4e2e-40b9-a5c3-28a1d9fe6220 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:39.600887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e39643a-4e2e-40b9-a5c3-28a1d9fe6220 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:39.688736 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e39643a-4e2e-40b9-a5c3-28a1d9fe6220 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:39.690242 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e39643a-4e2e-40b9-a5c3-28a1d9fe6220 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:39.809549 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:39.818343 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 344/690] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:19:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:19:39.823769 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cbb7b3-4671-423a-b0a6-c7d968861fd9 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:39.856256 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:39.947319 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cbb7b3-4671-423a-b0a6-c7d968861fd9 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:39.991376 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e7705-787d-4f3a-a845-30bfc2b266b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:39.992199 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e7705-787d-4f3a-a845-30bfc2b266b1 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:39.992501 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e7705-787d-4f3a-a845-30bfc2b266b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:39.992777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e7705-787d-4f3a-a845-30bfc2b266b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:40.006485 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e7705-787d-4f3a-a845-30bfc2b266b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:40.007950 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e7705-787d-4f3a-a845-30bfc2b266b1 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:40.027028 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-a11e7705-787d-4f3a-a845-30bfc2b266b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:40.027028 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:40.027028 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:40.027028 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:40.027028 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:40.027028 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:40.027028 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:40.027028 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:40.027028 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:40.027028 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:40.027028 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:40.027028 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:40.027028 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:40.027028 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:40.027028 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:40.027996 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:40.027996 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:40.027996 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:40.027996 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:40.027996 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:40.027996 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:40.027996 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:40.027996 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:40.027996 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:40.027996 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:40.027996 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:40.027996 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:40.027996 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:40.027996 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:40.027996 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:40.027996 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:40.027996 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:40.028993 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:40.028993 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:40.028993 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:40.028993 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:40.028993 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 347/691] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:40.034496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e39643a-4e2e-40b9-a5c3-28a1d9fe6220 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:40.190161 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e39643a-4e2e-40b9-a5c3-28a1d9fe6220 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:40.213336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de710ca-3620-49a0-abeb-8c16adbe7441 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:40.213618 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de710ca-3620-49a0-abeb-8c16adbe7441 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:40.213896 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de710ca-3620-49a0-abeb-8c16adbe7441 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:40.215473 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 348/692] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:40.220577 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520eeeae-9cd0-4e0b-8c22-14fb5a4ce054 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:40.220865 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520eeeae-9cd0-4e0b-8c22-14fb5a4ce054 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:40.221149 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520eeeae-9cd0-4e0b-8c22-14fb5a4ce054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:40.238473 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-520eeeae-9cd0-4e0b-8c22-14fb5a4ce054 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:40.240182 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-520eeeae-9cd0-4e0b-8c22-14fb5a4ce054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:40.242815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a96ba-0c51-4856-b2b1-328219fecfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:40.243663 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a96ba-0c51-4856-b2b1-328219fecfe7 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:40.243953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a96ba-0c51-4856-b2b1-328219fecfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:40.244263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a96ba-0c51-4856-b2b1-328219fecfe7 None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:40.244997 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a96ba-0c51-4856-b2b1-328219fecfe7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:40.245311 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a96ba-0c51-4856-b2b1-328219fecfe7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:40.246696 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a96ba-0c51-4856-b2b1-328219fecfe7 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:40.269447 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 345/693] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:40.275393 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a96ba-0c51-4856-b2b1-328219fecfe7 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:40.367407 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 349/694] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 12:19:40.374968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:40.408368 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520eeeae-9cd0-4e0b-8c22-14fb5a4ce054 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:40.412534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:40.451102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a96ba-0c51-4856-b2b1-328219fecfe7 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:40.531693 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520eeeae-9cd0-4e0b-8c22-14fb5a4ce054 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:40.544905 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a4623-1740-400f-b96d-dbd3b8c7bebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:40.545772 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a4623-1740-400f-b96d-dbd3b8c7bebf None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:40.546004 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a4623-1740-400f-b96d-dbd3b8c7bebf None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:40.546288 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a4623-1740-400f-b96d-dbd3b8c7bebf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:40.565522 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184a4623-1740-400f-b96d-dbd3b8c7bebf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:40.567264 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184a4623-1740-400f-b96d-dbd3b8c7bebf None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:40.576641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa75802-b590-473a-81d3-5dbe4495cdc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:40.578731 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa75802-b590-473a-81d3-5dbe4495cdc4 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:40.580333 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa75802-b590-473a-81d3-5dbe4495cdc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:40.580814 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa75802-b590-473a-81d3-5dbe4495cdc4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:40.597612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-184a4623-1740-400f-b96d-dbd3b8c7bebf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:40.597612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:40.597612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:40.597612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:40.597612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:40.597612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:40.597612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:40.597612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:40.597612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:40.597612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:40.597612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:40.597612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:40.597612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:19:40.597612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:19:40.597612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:19:40.598555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:19:40.598555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:40.598555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:40.598555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:40.598555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:40.598555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:40.598555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:40.598555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:40.598555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:40.598555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:40.598555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:40.598555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:40.598555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:40.598555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:40.598555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:40.598555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:40.598555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:40.600826 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:19:40.600826 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:19:40.600826 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:19:40.600826 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:19:40.600826 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:19:40.600826 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:19:40.600826 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:40.600826 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:40.607770 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 346/695] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:40.608646 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa75802-b590-473a-81d3-5dbe4495cdc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:40.613632 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a4623-1740-400f-b96d-dbd3b8c7bebf None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:40.628180 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-8fa75802-b590-473a-81d3-5dbe4495cdc4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:19:40.628180 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:40.628180 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:40.628180 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:40.628180 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:40.628180 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:40.628180 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:40.628180 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:40.628180 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:40.628180 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:40.628180 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:40.628180 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:40.628180 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:40.628180 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:40.628180 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:40.628180 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:40.629197 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:40.629197 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:40.629197 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:40.629197 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:40.629197 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:40.629197 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:40.629197 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:40.629197 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:40.629197 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:40.629197 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:40.629197 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:40.629197 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:40.629197 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:40.629197 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:40.629197 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:40.629197 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:40.630130 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:40.630130 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:40.630130 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:40.630130 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:40.630130 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:40.630130 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:40.630130 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:40.630130 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:40.630130 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:40.630130 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:40.630130 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:19:40.630130 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:40.633181 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 350/696] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:19:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:19:40.639336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa75802-b590-473a-81d3-5dbe4495cdc4 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:40.722552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:40.748003 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a4623-1740-400f-b96d-dbd3b8c7bebf None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:40.763267 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa75802-b590-473a-81d3-5dbe4495cdc4 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:40.769133 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:40.798217 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba532c8-7595-4d5a-aff4-21287ab4cda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:40.798452 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bfd0a-6115-4611-a0db-39ab2a4a7eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:40.798892 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba532c8-7595-4d5a-aff4-21287ab4cda7 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:40.799177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba532c8-7595-4d5a-aff4-21287ab4cda7 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:40.799455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba532c8-7595-4d5a-aff4-21287ab4cda7 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:40.800034 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bfd0a-6115-4611-a0db-39ab2a4a7eee None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:40.800245 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba532c8-7595-4d5a-aff4-21287ab4cda7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:40.800537 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba532c8-7595-4d5a-aff4-21287ab4cda7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:40.800958 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bfd0a-6115-4611-a0db-39ab2a4a7eee None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:40.801424 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bfd0a-6115-4611-a0db-39ab2a4a7eee None admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:40.802414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba532c8-7595-4d5a-aff4-21287ab4cda7 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:40.803569 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093bfd0a-6115-4611-a0db-39ab2a4a7eee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:40.804152 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093bfd0a-6115-4611-a0db-39ab2a4a7eee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:40.805900 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093bfd0a-6115-4611-a0db-39ab2a4a7eee None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:40.812456 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 351/697] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:19:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:40.816327 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba532c8-7595-4d5a-aff4-21287ab4cda7 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:40.838881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 347/698] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:19:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:19:40.844467 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bfd0a-6115-4611-a0db-39ab2a4a7eee None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:40.981162 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba532c8-7595-4d5a-aff4-21287ab4cda7 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:41.010770 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:41.019070 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d3f06-92da-4308-8984-07d74334fe07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:41.020137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d3f06-92da-4308-8984-07d74334fe07 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:41.020452 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d3f06-92da-4308-8984-07d74334fe07 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:41.020763 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d3f06-92da-4308-8984-07d74334fe07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:41.033579 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d3f06-92da-4308-8984-07d74334fe07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:41.035072 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d3f06-92da-4308-8984-07d74334fe07 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:41.057051 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-c59d3f06-92da-4308-8984-07d74334fe07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:41.057051 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:41.057051 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:41.057051 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:41.057051 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:41.057051 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:41.057051 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:41.057051 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:41.057051 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:41.057051 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:41.057051 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:41.057051 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:41.057051 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:41.057051 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:41.057051 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:41.058017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:41.058017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:41.058017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:41.058017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:41.058017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:41.058017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:41.058017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:41.058017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:41.058017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:41.058017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:41.058017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:41.058017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:41.058017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:41.058017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:41.058017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:41.058017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:41.058017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:41.058913 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:41.058913 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:41.058913 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:41.058913 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:41.058913 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 352/699] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:41.064925 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d3f06-92da-4308-8984-07d74334fe07 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:41.083228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bfd0a-6115-4611-a0db-39ab2a4a7eee None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:41.145359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10216492-acd3-4e41-9f96-acd084951533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:41.146589 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10216492-acd3-4e41-9f96-acd084951533 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:41.146873 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10216492-acd3-4e41-9f96-acd084951533 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:41.147151 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10216492-acd3-4e41-9f96-acd084951533 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:41.147988 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10216492-acd3-4e41-9f96-acd084951533 None admin] RBAC: Adding query filter params (role.id=81d82471e2be4f0aa8164b75c16a9718, scope.project.id=839585501f414e1ab02501422510e54f, user.id=71fb9a79ac494635804b93d4a0048205) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:41.148333 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10216492-acd3-4e41-9f96-acd084951533 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:41.149779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10216492-acd3-4e41-9f96-acd084951533 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:41.176255 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 348/700] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:19:40 2022] GET /identity/v3/role_assignments?user.id=71fb9a79ac494635804b93d4a0048205&scope.project.id=839585501f414e1ab02501422510e54f&role.id=81d82471e2be4f0aa8164b75c16a9718 => generated 608 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:19:41.236834 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c492a7b-d68d-4512-b568-efec2c2e1b61 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:41.237449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c492a7b-d68d-4512-b568-efec2c2e1b61 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:41.237929 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c492a7b-d68d-4512-b568-efec2c2e1b61 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:41.240023 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 349/701] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:19:41.251889 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b200c37-749b-4371-995a-a58186175d2c None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:41.252223 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b200c37-749b-4371-995a-a58186175d2c None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:41.252512 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b200c37-749b-4371-995a-a58186175d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:41.268773 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d3f06-92da-4308-8984-07d74334fe07 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:41.290719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-4b200c37-749b-4371-995a-a58186175d2c None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:41.291964 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-4b200c37-749b-4371-995a-a58186175d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:41.337395 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843791cf-9e87-42d3-b5b4-ab4f35417191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:41.338084 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843791cf-9e87-42d3-b5b4-ab4f35417191 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:41.338370 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843791cf-9e87-42d3-b5b4-ab4f35417191 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:41.338652 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843791cf-9e87-42d3-b5b4-ab4f35417191 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:41.339390 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843791cf-9e87-42d3-b5b4-ab4f35417191 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:41.339706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843791cf-9e87-42d3-b5b4-ab4f35417191 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:41.341143 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843791cf-9e87-42d3-b5b4-ab4f35417191 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:41.367714 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:41.373799 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 353/702] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:41.379648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843791cf-9e87-42d3-b5b4-ab4f35417191 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:41.413203 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 350/703] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 12:19:41.427765 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:41.438215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b200c37-749b-4371-995a-a58186175d2c None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:41.509401 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843791cf-9e87-42d3-b5b4-ab4f35417191 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:41.559248 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b68527-2ef3-4c7f-ae6d-60bdad830fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:41.560438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b68527-2ef3-4c7f-ae6d-60bdad830fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:41.560928 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b68527-2ef3-4c7f-ae6d-60bdad830fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:41.561470 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b68527-2ef3-4c7f-ae6d-60bdad830fc5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:41.572193 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b200c37-749b-4371-995a-a58186175d2c None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:41.572400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b68527-2ef3-4c7f-ae6d-60bdad830fc5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:41.572400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b68527-2ef3-4c7f-ae6d-60bdad830fc5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:41.587831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-11b68527-2ef3-4c7f-ae6d-60bdad830fc5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 12:19:41.587831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:41.587831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:41.587831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:41.587831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:41.587831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:41.587831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:41.587831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:41.587831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:41.587831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:41.587831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:41.587831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:41.587831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:19:41.587831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:19:41.587831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:19:41.587831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:19:41.588856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:41.588856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:41.588856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:19:41.588856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:41.588856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:19:41.588856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:41.588856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:41.588856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:41.588856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:41.588856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:41.588856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:41.588856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:41.588856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:41.588856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:41.588856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:41.588856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:41.588856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:41.588856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:41.590292 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:41.590292 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:41.590292 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:19:41.590292 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:19:41.590292 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:19:41.590292 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:19:41.590292 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:19:41.590292 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:19:41.590292 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 12:19:41.590292 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:41.590292 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 354/704] 10.222.0.16 () {56 vars in 1093 bytes} [Mon May 23 12:19:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:41.605562 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b68527-2ef3-4c7f-ae6d-60bdad830fc5 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:41.614567 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549166aa-50c4-498f-826e-edce7c7ce400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:41.615616 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549166aa-50c4-498f-826e-edce7c7ce400 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:41.615616 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549166aa-50c4-498f-826e-edce7c7ce400 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:41.616199 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549166aa-50c4-498f-826e-edce7c7ce400 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:41.616762 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549166aa-50c4-498f-826e-edce7c7ce400 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:41.618731 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549166aa-50c4-498f-826e-edce7c7ce400 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:41.639709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-549166aa-50c4-498f-826e-edce7c7ce400 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 12:19:41.639709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:41.639709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 12:19:41.639709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 12:19:41.639709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 12:19:41.639709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 12:19:41.639709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 12:19:41.639709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:19:41.639709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 12:19:41.639709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:41.639709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:19:41.639709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:41.639709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:41.639709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:41.639709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:41.639709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:41.642044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:41.642044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:41.642044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:41.642044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:41.642044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:41.642044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:41.642044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:41.642044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 12:19:41.642044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 12:19:41.642044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 12:19:41.642044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 12:19:41.642044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:41.642044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:41.642044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:41.642044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:41.642044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:41.644102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:41.644102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:41.644102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:41.644102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:41.644102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:41.644102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:41.644102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:41.644102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:41.644102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:41.644102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:41.644102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:41.644102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 12:19:41.644102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:19:41.644102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 12:19:41.644102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:41.644102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 351/705] 10.222.0.16 () {56 vars in 1129 bytes} [Mon May 23 12:19:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:41.649661 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549166aa-50c4-498f-826e-edce7c7ce400 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:41.698916 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:41.733211 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:41.738699 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b68527-2ef3-4c7f-ae6d-60bdad830fc5 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:41.788402 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15663fc5-ab54-4f31-8fab-d68daa4fea83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:41.788992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15663fc5-ab54-4f31-8fab-d68daa4fea83 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:41.789278 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15663fc5-ab54-4f31-8fab-d68daa4fea83 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:41.789558 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15663fc5-ab54-4f31-8fab-d68daa4fea83 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:41.789831 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549166aa-50c4-498f-826e-edce7c7ce400 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:41.790398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15663fc5-ab54-4f31-8fab-d68daa4fea83 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:41.790709 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15663fc5-ab54-4f31-8fab-d68daa4fea83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:41.792457 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15663fc5-ab54-4f31-8fab-d68daa4fea83 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:41.839468 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 355/706] 10.222.0.16 () {56 vars in 1133 bytes} [Mon May 23 12:19:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:41.845414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15663fc5-ab54-4f31-8fab-d68daa4fea83 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:41.859919 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c2c8f-2fc8-4d79-997a-595239f990f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:41.860976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c2c8f-2fc8-4d79-997a-595239f990f9 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:41.861485 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c2c8f-2fc8-4d79-997a-595239f990f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:41.862109 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c2c8f-2fc8-4d79-997a-595239f990f9 None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:41.864872 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c2c8f-2fc8-4d79-997a-595239f990f9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:41.865935 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c2c8f-2fc8-4d79-997a-595239f990f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:41.869197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c2c8f-2fc8-4d79-997a-595239f990f9 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:41.879139 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 352/707] 10.222.0.16 () {56 vars in 1124 bytes} [Mon May 23 12:19:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:41.886072 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c2c8f-2fc8-4d79-997a-595239f990f9 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:41.996401 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15663fc5-ab54-4f31-8fab-d68daa4fea83 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:42.059343 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec17c3-ecd5-4707-a1e8-6ce2a5835db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:42.059942 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec17c3-ecd5-4707-a1e8-6ce2a5835db9 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:42.060250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec17c3-ecd5-4707-a1e8-6ce2a5835db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:42.060534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec17c3-ecd5-4707-a1e8-6ce2a5835db9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:42.100544 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:42.105628 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c2c8f-2fc8-4d79-997a-595239f990f9 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:42.125524 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:42.147109 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b4446-9759-49d3-8fab-28f492ddbd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:42.147781 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b4446-9759-49d3-8fab-28f492ddbd82 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:42.148067 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b4446-9759-49d3-8fab-28f492ddbd82 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:42.148382 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b4446-9759-49d3-8fab-28f492ddbd82 None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:42.149094 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200b4446-9759-49d3-8fab-28f492ddbd82 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:42.149405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200b4446-9759-49d3-8fab-28f492ddbd82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:42.150781 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200b4446-9759-49d3-8fab-28f492ddbd82 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:42.157299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 353/708] 10.222.0.16 () {56 vars in 1124 bytes} [Mon May 23 12:19:41 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:19:42.199928 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec17c3-ecd5-4707-a1e8-6ce2a5835db9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:42.201442 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec17c3-ecd5-4707-a1e8-6ce2a5835db9 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:42.285406 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c3990-1db5-4985-9eff-a97dc2e5cdf8 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:42.286055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c3990-1db5-4985-9eff-a97dc2e5cdf8 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:42.286577 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c3990-1db5-4985-9eff-a97dc2e5cdf8 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:42.292367 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 354/709] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:42.294503 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af83ec-193f-4b05-a328-bb7eacfab1ba None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:42.295056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af83ec-193f-4b05-a328-bb7eacfab1ba None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:42.295552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af83ec-193f-4b05-a328-bb7eacfab1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:42.314447 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 356/710] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:19:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:19:42.322719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec17c3-ecd5-4707-a1e8-6ce2a5835db9 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:42.334559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-14af83ec-193f-4b05-a328-bb7eacfab1ba None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:42.335790 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-14af83ec-193f-4b05-a328-bb7eacfab1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:42.433496 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:42.490025 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec17c3-ecd5-4707-a1e8-6ce2a5835db9 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:42.530167 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65560db5-0a92-4484-9d70-5cb767b6301f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:42.531372 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65560db5-0a92-4484-9d70-5cb767b6301f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:42.531857 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65560db5-0a92-4484-9d70-5cb767b6301f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:42.532404 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65560db5-0a92-4484-9d70-5cb767b6301f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:42.545322 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65560db5-0a92-4484-9d70-5cb767b6301f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:42.547236 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65560db5-0a92-4484-9d70-5cb767b6301f None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:42.571203 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-65560db5-0a92-4484-9d70-5cb767b6301f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:42.571203 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:42.571203 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:42.571203 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:42.571203 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:42.571203 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:42.571203 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:42.571203 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:42.571203 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:42.571203 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:42.571203 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:42.571203 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:42.571203 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:42.571203 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:42.571203 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:42.572163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:42.572163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:42.572163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:42.572163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:42.572163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:42.572163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:42.572163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:42.572163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:42.572163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:42.572163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:42.572163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:42.572163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:42.572163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:42.572163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:42.572163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:42.572163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:42.572163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:42.573124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:42.573124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:42.573124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:42.573124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:42.574002 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 357/711] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:42.575064 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa02324-c2ca-4923-942d-a8a863a644ba None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:42.575350 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa02324-c2ca-4923-942d-a8a863a644ba None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:42.575627 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa02324-c2ca-4923-942d-a8a863a644ba None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:42.580212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 358/712] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:42.582174 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e7b017-da98-451a-8789-9ce76052daa8 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:42.582461 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e7b017-da98-451a-8789-9ce76052daa8 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:42.582737 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e7b017-da98-451a-8789-9ce76052daa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:42.600094 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-92e7b017-da98-451a-8789-9ce76052daa8 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:42.600496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 355/713] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:42.601315 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-92e7b017-da98-451a-8789-9ce76052daa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:42.601814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af83ec-193f-4b05-a328-bb7eacfab1ba None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:42.684546 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 359/714] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:42.686189 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e7b017-da98-451a-8789-9ce76052daa8 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:42.730428 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:42.742775 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af83ec-193f-4b05-a328-bb7eacfab1ba None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:42.747579 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:42.781317 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72804c3-910d-4d38-ad16-f0822d45c16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:42.781984 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72804c3-910d-4d38-ad16-f0822d45c16c None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:42.782270 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72804c3-910d-4d38-ad16-f0822d45c16c None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:42.782552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72804c3-910d-4d38-ad16-f0822d45c16c None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:42.783281 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72804c3-910d-4d38-ad16-f0822d45c16c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:42.783655 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72804c3-910d-4d38-ad16-f0822d45c16c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:42.785011 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72804c3-910d-4d38-ad16-f0822d45c16c None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:42.802510 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 356/715] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:42.802510 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72804c3-910d-4d38-ad16-f0822d45c16c None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:42.841118 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e7b017-da98-451a-8789-9ce76052daa8 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:42.890062 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957effe-36f3-40ec-b149-4ae324a7da85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:42.890868 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957effe-36f3-40ec-b149-4ae324a7da85 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:42.891153 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957effe-36f3-40ec-b149-4ae324a7da85 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:42.891434 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957effe-36f3-40ec-b149-4ae324a7da85 None admin] PATH_INFO: `/v3/services/network` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:42.892097 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6957effe-36f3-40ec-b149-4ae324a7da85 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:42.893491 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6957effe-36f3-40ec-b149-4ae324a7da85 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:42.912780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72804c3-910d-4d38-ad16-f0822d45c16c None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:42.916298 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-6957effe-36f3-40ec-b149-4ae324a7da85 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 23 12:19:42.916298 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:42.916298 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 12:19:42.916298 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 12:19:42.916298 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 12:19:42.916298 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 12:19:42.916298 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 12:19:42.916298 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:19:42.916298 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 12:19:42.916298 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:42.916298 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:19:42.916298 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:42.916298 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:42.916298 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:42.916298 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:42.916298 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:42.916298 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:42.917395 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:42.917395 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:42.917395 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:42.917395 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:42.917395 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:42.917395 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:42.917395 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 12:19:42.917395 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 12:19:42.917395 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 12:19:42.917395 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 12:19:42.917395 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:42.917395 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:42.917395 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:42.917395 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:42.917395 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:42.917395 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:42.918250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:42.918250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:42.918250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:42.918250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:42.918250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:42.918250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:42.918250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:42.918250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:42.918250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:42.918250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:42.918250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 12:19:42.918250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:19:42.918250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 12:19:42.918250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:42.923050 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 360/716] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:42 2022] GET /identity/v3/services/network => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:42.923971 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957effe-36f3-40ec-b149-4ae324a7da85 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:42.953303 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef2216-1fca-46d6-8686-791c408e13e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:42.954296 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef2216-1fca-46d6-8686-791c408e13e7 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:42.954650 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef2216-1fca-46d6-8686-791c408e13e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:42.954998 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef2216-1fca-46d6-8686-791c408e13e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:42.966429 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ef2216-1fca-46d6-8686-791c408e13e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:42.974966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-46ef2216-1fca-46d6-8686-791c408e13e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:42.974966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:42.974966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:42.974966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:42.974966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:42.974966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:42.974966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:42.974966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:42.974966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:42.974966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:42.974966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:42.974966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:42.974966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:42.974966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:42.974966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:42.974966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:42.976877 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:42.976877 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:42.976877 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:42.976877 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:42.976877 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:42.976877 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:42.976877 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:42.976877 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:42.976877 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:42.976877 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:42.976877 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:42.976877 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:42.976877 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:42.976877 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:42.976877 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:42.976877 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:42.978730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:42.978730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:42.978730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:42.978730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:42.978730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:42.978730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:42.978730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:42.978730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:42.978730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:42.978730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:42.978730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:42.978730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:42.978730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 357/717] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 12:19:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:42.978730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef2216-1fca-46d6-8686-791c408e13e7 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:42.995199 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:43.060549 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957effe-36f3-40ec-b149-4ae324a7da85 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:43.063566 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:43.099801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43534c-e86d-4845-ae9b-48271d0b1df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:43.100522 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43534c-e86d-4845-ae9b-48271d0b1df9 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:43.100805 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43534c-e86d-4845-ae9b-48271d0b1df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:43.101078 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43534c-e86d-4845-ae9b-48271d0b1df9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:43.108631 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab43534c-e86d-4845-ae9b-48271d0b1df9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:43.110203 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab43534c-e86d-4845-ae9b-48271d0b1df9 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:43.117598 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef2216-1fca-46d6-8686-791c408e13e7 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:43.124042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-ab43534c-e86d-4845-ae9b-48271d0b1df9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:43.124042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:43.124042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:43.124042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:43.124042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:43.124042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:43.124042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:43.124042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:43.124042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:43.124042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:43.124042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:43.124042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:43.124042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:19:43.124042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:19:43.124042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:19:43.124976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:19:43.124976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:43.124976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:43.124976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:43.124976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:43.124976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:43.124976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:43.124976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:43.124976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:43.124976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:43.124976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:43.124976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:43.124976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:43.124976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:43.124976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:43.124976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:43.124976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:43.125865 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:19:43.125865 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:19:43.125865 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:19:43.125865 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:19:43.125865 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:19:43.125865 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:19:43.125865 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:43.125865 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:43.129844 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 361/718] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:43.130762 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43534c-e86d-4845-ae9b-48271d0b1df9 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:43.170053 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e7b33-2396-4762-9b07-fd09ceb5fd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:43.171092 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e7b33-2396-4762-9b07-fd09ceb5fd92 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:43.171578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e7b33-2396-4762-9b07-fd09ceb5fd92 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:43.172076 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5e7b33-2396-4762-9b07-fd09ceb5fd92 None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:43.173015 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e7b33-2396-4762-9b07-fd09ceb5fd92 None admin] RBAC: Adding query filter params (name=network) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:43.173522 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e7b33-2396-4762-9b07-fd09ceb5fd92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:43.175641 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5e7b33-2396-4762-9b07-fd09ceb5fd92 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:43.187763 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 358/719] 10.222.0.16 () {56 vars in 1112 bytes} [Mon May 23 12:19:42 2022] GET /identity/v3/services?name=network => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:43.187763 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e7b33-2396-4762-9b07-fd09ceb5fd92 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:43.259686 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43534c-e86d-4845-ae9b-48271d0b1df9 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:43.273900 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:43.301292 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae0f84-1b26-48a2-bf8c-b056029ac618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:43.302003 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae0f84-1b26-48a2-bf8c-b056029ac618 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:43.302284 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae0f84-1b26-48a2-bf8c-b056029ac618 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:43.302565 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae0f84-1b26-48a2-bf8c-b056029ac618 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:43.303438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae0f84-1b26-48a2-bf8c-b056029ac618 None admin] RBAC: Adding query filter params (name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:43.303749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae0f84-1b26-48a2-bf8c-b056029ac618 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:43.305397 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae0f84-1b26-48a2-bf8c-b056029ac618 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:43.309009 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5e7b33-2396-4762-9b07-fd09ceb5fd92 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:43.314320 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 362/720] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 12:19:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:43.315256 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae0f84-1b26-48a2-bf8c-b056029ac618 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:43.350271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d47a8-7970-48ee-8bdf-29b5426da743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:43.350847 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d47a8-7970-48ee-8bdf-29b5426da743 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:43.351193 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d47a8-7970-48ee-8bdf-29b5426da743 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:43.351478 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d47a8-7970-48ee-8bdf-29b5426da743 None admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:43.352344 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d47a8-7970-48ee-8bdf-29b5426da743 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:43.352660 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d47a8-7970-48ee-8bdf-29b5426da743 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:43.353348 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:43.354635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d47a8-7970-48ee-8bdf-29b5426da743 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:43.374713 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 359/721] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:19:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:43.375665 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d47a8-7970-48ee-8bdf-29b5426da743 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:43.453231 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae0f84-1b26-48a2-bf8c-b056029ac618 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:43.489979 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d47a8-7970-48ee-8bdf-29b5426da743 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:43.502438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4bb33-75c3-45b2-a962-b40adf9bf84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:43.504164 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4bb33-75c3-45b2-a962-b40adf9bf84b None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:43.504673 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4bb33-75c3-45b2-a962-b40adf9bf84b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:43.505372 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4bb33-75c3-45b2-a962-b40adf9bf84b None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:43.507192 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4bb33-75c3-45b2-a962-b40adf9bf84b None admin] RBAC: Adding query filter params (type=network) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:43.507706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4bb33-75c3-45b2-a962-b40adf9bf84b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:43.509780 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4bb33-75c3-45b2-a962-b40adf9bf84b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:43.524199 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 363/722] 10.222.0.16 () {56 vars in 1112 bytes} [Mon May 23 12:19:43 2022] GET /identity/v3/services?type=network => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:43.524199 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4bb33-75c3-45b2-a962-b40adf9bf84b None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:43.543918 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c58d0-865f-45fa-994e-7310ec46aa18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:43.544785 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c58d0-865f-45fa-994e-7310ec46aa18 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:43.545071 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c58d0-865f-45fa-994e-7310ec46aa18 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:43.545350 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72c58d0-865f-45fa-994e-7310ec46aa18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:43.549079 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:43.572679 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72c58d0-865f-45fa-994e-7310ec46aa18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:43.574139 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72c58d0-865f-45fa-994e-7310ec46aa18 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:43.618459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-a72c58d0-865f-45fa-994e-7310ec46aa18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:43.618459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:43.618459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:43.618459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:43.618459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:43.618459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:43.618459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:43.618459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:43.618459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:43.618459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:43.618459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:43.618459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:43.618459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:43.618459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:43.618459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:43.619440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:43.619440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:43.619440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:43.619440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:43.619440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:43.619440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:43.619440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:43.619440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:43.619440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:43.619440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:43.619440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:43.619440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:43.619440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:43.619440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:43.619440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:43.619440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:43.619440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:43.620938 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:43.620938 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:43.620938 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:43.620938 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:43.620938 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 360/723] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:43.624945 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c58d0-865f-45fa-994e-7310ec46aa18 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:43.659602 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:43.676868 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4bb33-75c3-45b2-a962-b40adf9bf84b None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:43.721612 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2a9de-08ad-4a07-b53a-7ee6cba61dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:43.722652 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2a9de-08ad-4a07-b53a-7ee6cba61dcb None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:43.723139 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2a9de-08ad-4a07-b53a-7ee6cba61dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:43.724820 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2a9de-08ad-4a07-b53a-7ee6cba61dcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:43.726019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f2a9de-08ad-4a07-b53a-7ee6cba61dcb None admin] RBAC: Adding query filter params (role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8, scope.project.id=839585501f414e1ab02501422510e54f, user.id=b83535b443a84275a160cd84cb3ad5a0) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:43.726637 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f2a9de-08ad-4a07-b53a-7ee6cba61dcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:43.728348 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f2a9de-08ad-4a07-b53a-7ee6cba61dcb None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:43.737276 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 364/724] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:19:43 2022] GET /identity/v3/role_assignments?user.id=b83535b443a84275a160cd84cb3ad5a0&scope.project.id=839585501f414e1ab02501422510e54f&role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:43.836903 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72c58d0-865f-45fa-994e-7310ec46aa18 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:43.879043 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b48b5-5a7f-4ce2-9da0-128ff3568c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:43.879644 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b48b5-5a7f-4ce2-9da0-128ff3568c9d None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:43.879970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b48b5-5a7f-4ce2-9da0-128ff3568c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:43.880284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b48b5-5a7f-4ce2-9da0-128ff3568c9d None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:43.881235 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824b48b5-5a7f-4ce2-9da0-128ff3568c9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:43.881552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824b48b5-5a7f-4ce2-9da0-128ff3568c9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:43.882980 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824b48b5-5a7f-4ce2-9da0-128ff3568c9d None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:43.897364 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 361/725] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:19:43.902936 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2a9de-08ad-4a07-b53a-7ee6cba61dcb None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:43.913495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169f509-2db7-49b3-ad6b-0fe12ffcb7cf None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:43.914171 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169f509-2db7-49b3-ad6b-0fe12ffcb7cf None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:43.914656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169f509-2db7-49b3-ad6b-0fe12ffcb7cf None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:43.916339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 362/726] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:43.926107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078c3f0-c733-48af-b6d8-1da7463deae0 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:43.926747 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078c3f0-c733-48af-b6d8-1da7463deae0 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:43.927406 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078c3f0-c733-48af-b6d8-1da7463deae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:43.954878 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-0078c3f0-c733-48af-b6d8-1da7463deae0 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:43.956470 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-0078c3f0-c733-48af-b6d8-1da7463deae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:44.045896 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2a9de-08ad-4a07-b53a-7ee6cba61dcb None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:44.048521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:44.093495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 363/727] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 12:19:44.115839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135328f-d4e8-457d-b486-218281ad0067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:44.116707 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135328f-d4e8-457d-b486-218281ad0067 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:44.116994 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135328f-d4e8-457d-b486-218281ad0067 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:44.117275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135328f-d4e8-457d-b486-218281ad0067 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:44.126007 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135328f-d4e8-457d-b486-218281ad0067 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:44.127452 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135328f-d4e8-457d-b486-218281ad0067 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:44.134285 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078c3f0-c733-48af-b6d8-1da7463deae0 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:44.155108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-e135328f-d4e8-457d-b486-218281ad0067 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 12:19:44.155108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:44.155108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:44.155108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:44.155108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:44.155108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:44.155108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:44.155108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:44.155108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:44.155108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:44.155108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:44.155108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:44.155108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:19:44.155108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:19:44.155108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:19:44.155108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:19:44.156112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:44.156112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:44.156112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:19:44.156112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:44.156112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:19:44.156112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:44.156112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:44.156112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:44.156112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:44.156112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:44.156112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:44.156112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:44.156112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:44.156112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:44.156112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:44.156112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:44.156112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:44.156112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:44.157100 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:44.157100 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:44.157100 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:19:44.157100 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:19:44.157100 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:19:44.157100 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:19:44.157100 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:19:44.157100 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:19:44.157100 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 12:19:44.157100 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:44.157100 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 365/728] 10.222.0.16 () {56 vars in 1117 bytes} [Mon May 23 12:19:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:44.162199 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135328f-d4e8-457d-b486-218281ad0067 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:44.178352 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:44.312946 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135328f-d4e8-457d-b486-218281ad0067 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:44.324203 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078c3f0-c733-48af-b6d8-1da7463deae0 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:44.363749 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5806c-8921-4c0a-95bd-05168cab3668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:44.365021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5806c-8921-4c0a-95bd-05168cab3668 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:44.365318 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5806c-8921-4c0a-95bd-05168cab3668 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:44.365615 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5806c-8921-4c0a-95bd-05168cab3668 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:44.368433 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be20a6-b018-4c96-bf1d-cd2d3604a19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:44.369364 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be20a6-b018-4c96-bf1d-cd2d3604a19f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:44.369842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be20a6-b018-4c96-bf1d-cd2d3604a19f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:44.370298 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be20a6-b018-4c96-bf1d-cd2d3604a19f None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:44.371332 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be20a6-b018-4c96-bf1d-cd2d3604a19f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:44.371830 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be20a6-b018-4c96-bf1d-cd2d3604a19f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:44.373535 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be20a6-b018-4c96-bf1d-cd2d3604a19f None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:44.394595 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 366/729] 10.222.0.16 () {56 vars in 1149 bytes} [Mon May 23 12:19:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:44.403139 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5806c-8921-4c0a-95bd-05168cab3668 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:44.403482 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be20a6-b018-4c96-bf1d-cd2d3604a19f None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:44.410962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-bda5806c-8921-4c0a-95bd-05168cab3668 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:19:44.410962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:44.410962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:44.410962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:44.410962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:44.410962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:44.410962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:44.410962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:44.410962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:44.410962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:44.410962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:44.410962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:44.410962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:44.410962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:44.410962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:44.410962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:44.412055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:44.412055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:44.412055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:44.412055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:44.412055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:44.412055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:44.412055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:44.412055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:44.412055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:44.412055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:44.412055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:44.412055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:44.412055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:44.412055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:44.412055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:44.412055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:44.413055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:44.413055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:44.413055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:44.413055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:44.413055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:44.413055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:44.413055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:44.413055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:44.413055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:44.413055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:44.413055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:19:44.413055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:44.421732 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 364/730] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:19:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:19:44.444126 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5806c-8921-4c0a-95bd-05168cab3668 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:44.498965 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:44.536774 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be20a6-b018-4c96-bf1d-cd2d3604a19f None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:44.550196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:44.579366 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17879bb2-5978-4cfb-b1a8-6136ff0f0bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:44.580193 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17879bb2-5978-4cfb-b1a8-6136ff0f0bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:44.580478 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17879bb2-5978-4cfb-b1a8-6136ff0f0bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:44.580765 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17879bb2-5978-4cfb-b1a8-6136ff0f0bf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:44.659765 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17879bb2-5978-4cfb-b1a8-6136ff0f0bf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:44.661270 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17879bb2-5978-4cfb-b1a8-6136ff0f0bf2 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:44.688087 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5806c-8921-4c0a-95bd-05168cab3668 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:44.726030 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b9e2ad-5ddc-42b6-8a8a-7c2e6d34b6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:44.727244 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9e2ad-5ddc-42b6-8a8a-7c2e6d34b6ba None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:44.728031 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9e2ad-5ddc-42b6-8a8a-7c2e6d34b6ba None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:44.728547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9e2ad-5ddc-42b6-8a8a-7c2e6d34b6ba None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:44.729498 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b9e2ad-5ddc-42b6-8a8a-7c2e6d34b6ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:44.730017 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b9e2ad-5ddc-42b6-8a8a-7c2e6d34b6ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:44.732404 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b9e2ad-5ddc-42b6-8a8a-7c2e6d34b6ba None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:44.744275 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 365/731] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:19:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:44.745887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06ea55-00da-4675-aa43-9af797585002 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:44.746429 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06ea55-00da-4675-aa43-9af797585002 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:44.746905 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06ea55-00da-4675-aa43-9af797585002 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:44.748661 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 366/732] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:44.750572 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06ea55-00da-4675-aa43-9af797585002 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:44.754583 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:44.794346 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 367/733] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:19:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 12:19:44.796042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f408d3d-278f-4953-b043-3deb93db8da2 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:44.796549 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f408d3d-278f-4953-b043-3deb93db8da2 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:44.797022 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f408d3d-278f-4953-b043-3deb93db8da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:44.814695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-5f408d3d-278f-4953-b043-3deb93db8da2 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:44.816356 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5f408d3d-278f-4953-b043-3deb93db8da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:44.859584 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:44.863935 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06ea55-00da-4675-aa43-9af797585002 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:44.892137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 368/734] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:44.893313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f408d3d-278f-4953-b043-3deb93db8da2 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:44.901309 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d4d49-4f4c-4268-9a19-e5c4a0808026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:44.902222 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d4d49-4f4c-4268-9a19-e5c4a0808026 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:44.902932 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d4d49-4f4c-4268-9a19-e5c4a0808026 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:44.902932 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d4d49-4f4c-4268-9a19-e5c4a0808026 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:44.914269 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d4d49-4f4c-4268-9a19-e5c4a0808026 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:44.915752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d4d49-4f4c-4268-9a19-e5c4a0808026 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:44.933479 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-ef6d4d49-4f4c-4268-9a19-e5c4a0808026 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:44.933479 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:44.933479 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:44.933479 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:44.933479 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:44.933479 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:44.933479 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:44.933479 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:44.933479 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:44.933479 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:44.933479 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:44.933479 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:44.933479 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:44.933479 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:44.933479 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:44.934541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:44.934541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:44.934541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:44.934541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:44.934541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:44.934541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:44.934541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:44.934541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:44.934541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:44.934541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:44.934541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:44.934541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:44.934541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:44.934541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:44.934541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:44.934541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:44.934541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:44.935468 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:44.935468 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:44.935468 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:44.935468 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:44.940238 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 367/735] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:44.941065 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d4d49-4f4c-4268-9a19-e5c4a0808026 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:45.003922 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:45.009102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f408d3d-278f-4953-b043-3deb93db8da2 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:45.048569 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d4d49-4f4c-4268-9a19-e5c4a0808026 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:45.049072 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0842f58d-a524-451f-824a-36c4949aa36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:45.050246 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842f58d-a524-451f-824a-36c4949aa36b None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:45.050770 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842f58d-a524-451f-824a-36c4949aa36b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:45.051225 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842f58d-a524-451f-824a-36c4949aa36b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:45.072828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0842f58d-a524-451f-824a-36c4949aa36b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:45.074412 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0842f58d-a524-451f-824a-36c4949aa36b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:45.091605 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d576d-560a-4094-aac4-17e864525230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:45.092574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d576d-560a-4094-aac4-17e864525230 None admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:45.093035 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d576d-560a-4094-aac4-17e864525230 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:45.093486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d576d-560a-4094-aac4-17e864525230 None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:45.094407 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d576d-560a-4094-aac4-17e864525230 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:45.096008 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d576d-560a-4094-aac4-17e864525230 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:45.097988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-0842f58d-a524-451f-824a-36c4949aa36b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:45.097988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:45.097988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:45.097988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:45.097988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:45.097988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:45.097988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:45.097988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:45.097988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:45.097988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:45.097988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:45.097988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:45.097988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:45.097988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:45.097988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:45.098909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:45.098909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:45.098909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:45.098909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:45.098909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:45.098909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:45.098909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:45.098909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:45.098909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:45.098909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:45.098909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:45.098909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:45.098909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:45.098909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:45.098909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:45.098909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:45.098909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:45.100206 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:45.100206 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:45.100206 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:45.100206 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:45.100206 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 369/736] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:45.100622 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0842f58d-a524-451f-824a-36c4949aa36b None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:45.119264 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 368/737] 10.222.0.16 () {60 vars in 1141 bytes} [Mon May 23 12:19:44 2022] POST /identity/v3/services => generated 251 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:19:45.120705 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d576d-560a-4094-aac4-17e864525230 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:45.142347 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:45.293557 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0842f58d-a524-451f-824a-36c4949aa36b None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:45.325101 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d576d-560a-4094-aac4-17e864525230 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:45.331938 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf16c0cb-9ec8-4b2f-9a0c-75302577a596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:45.332800 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16c0cb-9ec8-4b2f-9a0c-75302577a596 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:45.333279 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16c0cb-9ec8-4b2f-9a0c-75302577a596 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:45.333983 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16c0cb-9ec8-4b2f-9a0c-75302577a596 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:45.334898 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf16c0cb-9ec8-4b2f-9a0c-75302577a596 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:45.335354 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf16c0cb-9ec8-4b2f-9a0c-75302577a596 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:45.336909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf16c0cb-9ec8-4b2f-9a0c-75302577a596 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:45.349975 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 370/738] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:45.355599 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf16c0cb-9ec8-4b2f-9a0c-75302577a596 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:45.374545 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691fb4c-e214-47d6-b340-2e9dac9ac379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:45.375219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691fb4c-e214-47d6-b340-2e9dac9ac379 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:45.375505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691fb4c-e214-47d6-b340-2e9dac9ac379 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:45.375812 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691fb4c-e214-47d6-b340-2e9dac9ac379 None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:45.376252 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:45.376547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d691fb4c-e214-47d6-b340-2e9dac9ac379 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:45.376862 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d691fb4c-e214-47d6-b340-2e9dac9ac379 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:45.378296 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d691fb4c-e214-47d6-b340-2e9dac9ac379 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:45.392794 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 369/739] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:45.400188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691fb4c-e214-47d6-b340-2e9dac9ac379 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:45.469632 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf16c0cb-9ec8-4b2f-9a0c-75302577a596 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:45.472806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:45.510444 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ecc9f-d8f0-4f39-8f90-92b40448ca17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:45.510819 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691fb4c-e214-47d6-b340-2e9dac9ac379 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:45.511671 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ecc9f-d8f0-4f39-8f90-92b40448ca17 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:45.512125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ecc9f-d8f0-4f39-8f90-92b40448ca17 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:45.512720 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ecc9f-d8f0-4f39-8f90-92b40448ca17 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:45.521664 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ecc9f-d8f0-4f39-8f90-92b40448ca17 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:45.523416 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ecc9f-d8f0-4f39-8f90-92b40448ca17 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:45.540741 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-b02ecc9f-d8f0-4f39-8f90-92b40448ca17 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 12:19:45.540741 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:45.540741 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:45.540741 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:45.540741 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:45.540741 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:45.540741 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:45.540741 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:45.540741 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:45.540741 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:45.540741 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:45.540741 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:45.540741 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:19:45.540741 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:19:45.540741 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:19:45.540741 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:19:45.541930 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:45.541930 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:45.541930 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:19:45.541930 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:45.541930 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:19:45.541930 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:45.541930 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:45.541930 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:45.541930 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:45.541930 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:45.541930 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:45.541930 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:45.541930 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:45.541930 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:45.541930 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:45.541930 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:45.541930 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:45.541930 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:45.543098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:45.543098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:45.543098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:19:45.543098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:19:45.543098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:19:45.543098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:19:45.543098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:19:45.543098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:19:45.543098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 12:19:45.543098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:45.545523 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 371/740] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:19:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:45.548058 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8caa33-ae3c-44a2-ab16-928d766f801a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:45.548888 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8caa33-ae3c-44a2-ab16-928d766f801a None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:45.549176 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8caa33-ae3c-44a2-ab16-928d766f801a None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:45.549457 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8caa33-ae3c-44a2-ab16-928d766f801a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:45.552923 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ecc9f-d8f0-4f39-8f90-92b40448ca17 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:45.557806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8caa33-ae3c-44a2-ab16-928d766f801a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:45.559183 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8caa33-ae3c-44a2-ab16-928d766f801a None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:45.575267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-6c8caa33-ae3c-44a2-ab16-928d766f801a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:45.575267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:45.575267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:45.575267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:45.575267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:45.575267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:45.575267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:45.575267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:45.575267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:45.575267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:45.575267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:45.575267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:45.575267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:19:45.575267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:19:45.575267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:19:45.576265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:19:45.576265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:45.576265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:45.576265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:45.576265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:45.576265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:45.576265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:45.576265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:45.576265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:45.576265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:45.576265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:45.576265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:45.576265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:45.576265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:45.576265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:45.576265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:45.576265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:45.577220 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:19:45.577220 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:19:45.577220 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:19:45.577220 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:19:45.577220 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:19:45.577220 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:19:45.577220 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:45.577220 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:45.577220 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 370/741] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:45.582712 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8caa33-ae3c-44a2-ab16-928d766f801a None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:45.633369 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:45.667599 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ecc9f-d8f0-4f39-8f90-92b40448ca17 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:45.692452 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8caa33-ae3c-44a2-ab16-928d766f801a None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:45.706128 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b2be6-c7f0-403f-8d15-af3c86cbef67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:45.706760 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b2be6-c7f0-403f-8d15-af3c86cbef67 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:45.707046 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b2be6-c7f0-403f-8d15-af3c86cbef67 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:45.707325 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b2be6-c7f0-403f-8d15-af3c86cbef67 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:45.708495 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b2be6-c7f0-403f-8d15-af3c86cbef67 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:45.708818 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b2be6-c7f0-403f-8d15-af3c86cbef67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:45.710250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b2be6-c7f0-403f-8d15-af3c86cbef67 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:45.722528 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:45.732464 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04fe36-6ad1-4a55-9a05-cbceb6be56d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:45.733435 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04fe36-6ad1-4a55-9a05-cbceb6be56d8 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:45.733919 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 372/742] 10.222.0.16 () {56 vars in 1135 bytes} [Mon May 23 12:19:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:19:45.734162 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04fe36-6ad1-4a55-9a05-cbceb6be56d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:45.734716 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04fe36-6ad1-4a55-9a05-cbceb6be56d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:45.735772 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04fe36-6ad1-4a55-9a05-cbceb6be56d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:45.736333 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04fe36-6ad1-4a55-9a05-cbceb6be56d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:45.738060 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04fe36-6ad1-4a55-9a05-cbceb6be56d8 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:45.741413 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b2be6-c7f0-403f-8d15-af3c86cbef67 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:45.754112 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 371/743] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:19:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:19:45.760641 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04fe36-6ad1-4a55-9a05-cbceb6be56d8 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:45.893093 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b2be6-c7f0-403f-8d15-af3c86cbef67 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:45.919346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04fe36-6ad1-4a55-9a05-cbceb6be56d8 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:45.949864 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:45.963715 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c483348-9aaf-4a25-b0cd-8589ffa523b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:45.965957 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c483348-9aaf-4a25-b0cd-8589ffa523b0 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:45.966452 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c483348-9aaf-4a25-b0cd-8589ffa523b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:45.967001 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c483348-9aaf-4a25-b0cd-8589ffa523b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:45.984715 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2b5b9-568c-4d79-a609-0a20dd421a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:45.985621 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2b5b9-568c-4d79-a609-0a20dd421a02 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:45.986074 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2b5b9-568c-4d79-a609-0a20dd421a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:45.986503 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2b5b9-568c-4d79-a609-0a20dd421a02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:45.988158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c2b5b9-568c-4d79-a609-0a20dd421a02 None admin] RBAC: Adding query filter params (role.id=81d82471e2be4f0aa8164b75c16a9718, scope.project.id=839585501f414e1ab02501422510e54f, user.id=733f8296ec3d4795a1140c1e1cb6eb95) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:45.991134 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c2b5b9-568c-4d79-a609-0a20dd421a02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:45.992794 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c2b5b9-568c-4d79-a609-0a20dd421a02 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:46.003798 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 372/744] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:19:45 2022] GET /identity/v3/role_assignments?user.id=733f8296ec3d4795a1140c1e1cb6eb95&scope.project.id=839585501f414e1ab02501422510e54f&role.id=81d82471e2be4f0aa8164b75c16a9718 => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:19:46.069193 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c483348-9aaf-4a25-b0cd-8589ffa523b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:46.071672 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c483348-9aaf-4a25-b0cd-8589ffa523b0 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:46.187775 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 373/745] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:19:45 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 12:19:46.194703 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2b5b9-568c-4d79-a609-0a20dd421a02 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:46.293130 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:46.358321 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2b5b9-568c-4d79-a609-0a20dd421a02 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:46.373916 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f09dc-f547-4edc-aec6-72e7527374b0 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:46.374441 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f09dc-f547-4edc-aec6-72e7527374b0 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:46.374633 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f09dc-f547-4edc-aec6-72e7527374b0 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:46.375860 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 374/746] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:19:46.382312 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8b76c-b189-47a5-a20d-e4ae1c735437 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:46.382605 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8b76c-b189-47a5-a20d-e4ae1c735437 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:46.382893 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8b76c-b189-47a5-a20d-e4ae1c735437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:46.406972 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56bfee1-2ffd-41d9-a6aa-79334179e375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:46.407781 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56bfee1-2ffd-41d9-a6aa-79334179e375 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:46.408068 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56bfee1-2ffd-41d9-a6aa-79334179e375 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:46.408377 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56bfee1-2ffd-41d9-a6aa-79334179e375 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:46.410249 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-35e8b76c-b189-47a5-a20d-e4ae1c735437 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:46.420118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56bfee1-2ffd-41d9-a6aa-79334179e375 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:46.420278 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-35e8b76c-b189-47a5-a20d-e4ae1c735437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:46.421607 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56bfee1-2ffd-41d9-a6aa-79334179e375 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:46.439756 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-e56bfee1-2ffd-41d9-a6aa-79334179e375 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:46.439756 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:46.439756 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:46.439756 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:46.439756 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:46.439756 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:46.439756 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:46.439756 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:46.439756 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:46.439756 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:46.439756 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:46.439756 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:46.439756 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:46.439756 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:46.439756 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:46.440709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:46.440709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:46.440709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:46.440709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:46.440709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:46.440709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:46.440709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:46.440709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:46.440709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:46.440709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:46.440709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:46.440709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:46.440709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:46.440709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:46.440709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:46.440709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:46.440709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:46.443042 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:46.443042 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:46.443042 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:46.443042 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:46.443042 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 373/747] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:19:46.449323 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56bfee1-2ffd-41d9-a6aa-79334179e375 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:46.465466 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:46.536836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 375/748] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:46 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:46.538798 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498803c5-b7e2-49f6-a98b-6b76144f210e None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:46.539093 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498803c5-b7e2-49f6-a98b-6b76144f210e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:46.539376 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498803c5-b7e2-49f6-a98b-6b76144f210e None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:46.541034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 376/749] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:46.546523 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387628d-9fb9-45f5-a366-72e8d4362c88 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:46.546814 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387628d-9fb9-45f5-a366-72e8d4362c88 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:46.547099 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6387628d-9fb9-45f5-a366-72e8d4362c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:46.621665 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56bfee1-2ffd-41d9-a6aa-79334179e375 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:46.624565 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:46.668864 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e644a-a916-4f11-9b63-a9da60251527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:46.669938 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e644a-a916-4f11-9b63-a9da60251527 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:46.670428 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e644a-a916-4f11-9b63-a9da60251527 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:46.670949 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e644a-a916-4f11-9b63-a9da60251527 None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:46.671906 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2e644a-a916-4f11-9b63-a9da60251527 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:46.672445 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2e644a-a916-4f11-9b63-a9da60251527 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:46.674061 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2e644a-a916-4f11-9b63-a9da60251527 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:46.693611 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 374/750] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:46.694961 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e644a-a916-4f11-9b63-a9da60251527 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:46.698053 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-6387628d-9fb9-45f5-a366-72e8d4362c88 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:46.699451 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-6387628d-9fb9-45f5-a366-72e8d4362c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:46.771189 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 377/751] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:46 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 12:19:46.772336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387628d-9fb9-45f5-a366-72e8d4362c88 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:46.825166 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e644a-a916-4f11-9b63-a9da60251527 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:46.870890 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f679ac3c-8663-40dd-a224-51ad301661b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:46.871965 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679ac3c-8663-40dd-a224-51ad301661b8 None admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:46.872729 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679ac3c-8663-40dd-a224-51ad301661b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:46.873167 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679ac3c-8663-40dd-a224-51ad301661b8 None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:46.874344 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f679ac3c-8663-40dd-a224-51ad301661b8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:46.876505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f679ac3c-8663-40dd-a224-51ad301661b8 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:46.876641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:46.880907 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6387628d-9fb9-45f5-a366-72e8d4362c88 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:46.894499 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 375/752] 10.222.0.16 () {60 vars in 1141 bytes} [Mon May 23 12:19:46 2022] POST /identity/v3/services => generated 240 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:19:46.896600 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f679ac3c-8663-40dd-a224-51ad301661b8 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:46.935043 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:46.935722 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19760265-2683-4ac2-aa44-02bd3e0184ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:46.936124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19760265-2683-4ac2-aa44-02bd3e0184ed None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:46.936686 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19760265-2683-4ac2-aa44-02bd3e0184ed None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:46.936918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19760265-2683-4ac2-aa44-02bd3e0184ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:46.947927 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19760265-2683-4ac2-aa44-02bd3e0184ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:46.949647 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19760265-2683-4ac2-aa44-02bd3e0184ed None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:46.965129 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-19760265-2683-4ac2-aa44-02bd3e0184ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:46.965129 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:46.965129 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:46.965129 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:46.965129 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:46.965129 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:46.965129 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:46.965129 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:46.965129 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:46.965129 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:46.965129 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:46.965129 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:46.965129 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:19:46.965129 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:19:46.965129 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:19:46.966300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:19:46.966300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:46.966300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:46.966300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:46.966300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:46.966300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:46.966300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:46.966300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:46.966300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:46.966300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:46.966300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:46.966300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:46.966300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:46.966300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:46.966300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:46.966300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:46.966300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:46.967414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:19:46.967414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:19:46.967414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:19:46.967414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:19:46.967414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:19:46.967414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:19:46.967414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:46.967414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:46.974133 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 378/753] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:46.976062 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19760265-2683-4ac2-aa44-02bd3e0184ed None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:47.041591 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f679ac3c-8663-40dd-a224-51ad301661b8 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:47.118130 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe875b74-6a08-4713-9d1b-c34de407aa19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:47.119248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe875b74-6a08-4713-9d1b-c34de407aa19 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:47.119680 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe875b74-6a08-4713-9d1b-c34de407aa19 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:47.120120 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe875b74-6a08-4713-9d1b-c34de407aa19 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:47.121154 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19760265-2683-4ac2-aa44-02bd3e0184ed None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:47.132078 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe875b74-6a08-4713-9d1b-c34de407aa19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:47.139904 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-fe875b74-6a08-4713-9d1b-c34de407aa19 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:19:47.139904 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:47.139904 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:47.139904 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:47.139904 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:47.139904 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:47.139904 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:47.139904 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:47.139904 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:47.139904 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:47.139904 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:47.139904 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:47.139904 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:47.139904 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:47.139904 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:47.139904 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:47.141955 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:47.141955 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:47.141955 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:47.141955 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:47.141955 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:47.141955 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:47.141955 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:47.141955 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:47.141955 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:47.141955 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:47.141955 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:47.141955 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:47.141955 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:47.141955 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:47.141955 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:47.141955 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:47.143013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:47.143013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:47.143013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:47.143013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:47.143013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:47.143013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:47.143013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:47.143013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:47.143013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:47.143013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:47.143013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:19:47.143013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:47.145349 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 376/754] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:19:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:47.151155 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe875b74-6a08-4713-9d1b-c34de407aa19 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:47.157973 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21873334-7a62-4aa4-a7d5-6c9926055a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:47.158644 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21873334-7a62-4aa4-a7d5-6c9926055a02 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:47.158929 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21873334-7a62-4aa4-a7d5-6c9926055a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:47.159206 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21873334-7a62-4aa4-a7d5-6c9926055a02 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:47.159998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21873334-7a62-4aa4-a7d5-6c9926055a02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:47.160348 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21873334-7a62-4aa4-a7d5-6c9926055a02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:47.162516 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21873334-7a62-4aa4-a7d5-6c9926055a02 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:47.180153 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 379/755] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:19:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:47.183828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21873334-7a62-4aa4-a7d5-6c9926055a02 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:47.197605 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:47.276980 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:47.302612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe875b74-6a08-4713-9d1b-c34de407aa19 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:47.316822 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21873334-7a62-4aa4-a7d5-6c9926055a02 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:47.351026 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1464841-9d6a-48e7-9bad-d428aabe2c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:47.352069 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1464841-9d6a-48e7-9bad-d428aabe2c5a None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:47.352626 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1464841-9d6a-48e7-9bad-d428aabe2c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:47.353560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1464841-9d6a-48e7-9bad-d428aabe2c5a None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:47.354779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1464841-9d6a-48e7-9bad-d428aabe2c5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:47.355296 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1464841-9d6a-48e7-9bad-d428aabe2c5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:47.356903 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1464841-9d6a-48e7-9bad-d428aabe2c5a None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:47.357872 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbefb4fc-aab3-4aaa-8d52-6157ca6d5485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:47.358538 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefb4fc-aab3-4aaa-8d52-6157ca6d5485 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:47.358834 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefb4fc-aab3-4aaa-8d52-6157ca6d5485 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:47.359165 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbefb4fc-aab3-4aaa-8d52-6157ca6d5485 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:47.360040 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbefb4fc-aab3-4aaa-8d52-6157ca6d5485 None admin] RBAC: Adding query filter params (role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8, scope.project.id=839585501f414e1ab02501422510e54f, user.id=71fb9a79ac494635804b93d4a0048205) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:47.360399 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbefb4fc-aab3-4aaa-8d52-6157ca6d5485 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:47.361938 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbefb4fc-aab3-4aaa-8d52-6157ca6d5485 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:47.368867 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 377/757] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:19:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:47.376219 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 380/757] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:19:47 2022] GET /identity/v3/role_assignments?user.id=71fb9a79ac494635804b93d4a0048205&scope.project.id=839585501f414e1ab02501422510e54f&role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:47.378017 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1464841-9d6a-48e7-9bad-d428aabe2c5a None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:47.507962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1464841-9d6a-48e7-9bad-d428aabe2c5a None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:47.550541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e073df8-5a52-4615-9182-e9bfe8ed9877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:47.552263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e073df8-5a52-4615-9182-e9bfe8ed9877 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:47.552789 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e073df8-5a52-4615-9182-e9bfe8ed9877 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:47.553563 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e073df8-5a52-4615-9182-e9bfe8ed9877 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:47.568731 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e073df8-5a52-4615-9182-e9bfe8ed9877 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:47.570640 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e073df8-5a52-4615-9182-e9bfe8ed9877 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:47.578953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:47.591244 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-9e073df8-5a52-4615-9182-e9bfe8ed9877 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:47.591244 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:47.591244 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:47.591244 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:47.591244 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:47.591244 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:47.591244 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:47.591244 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:47.591244 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:47.591244 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:47.591244 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:47.591244 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:47.591244 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:47.591244 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:47.591244 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:47.592339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:47.592339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:47.592339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:47.592339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:47.592339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:47.592339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:47.592339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:47.592339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:47.592339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:47.592339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:47.592339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:47.592339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:47.592339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:47.592339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:47.592339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:47.592339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:47.592339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:47.593249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:47.593249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:47.593249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:47.593249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:47.593249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 378/758] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:47.599467 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbefb4fc-aab3-4aaa-8d52-6157ca6d5485 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:47.695353 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:47.729641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbefb4fc-aab3-4aaa-8d52-6157ca6d5485 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:47.776374 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa16ed28-2d9a-4533-806d-8b4be6967e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:47.777046 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16ed28-2d9a-4533-806d-8b4be6967e21 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:47.777333 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16ed28-2d9a-4533-806d-8b4be6967e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:47.777611 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa16ed28-2d9a-4533-806d-8b4be6967e21 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:47.778362 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa16ed28-2d9a-4533-806d-8b4be6967e21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:47.778668 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa16ed28-2d9a-4533-806d-8b4be6967e21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:47.780036 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa16ed28-2d9a-4533-806d-8b4be6967e21 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:47.799423 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 381/759] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:47.804909 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e073df8-5a52-4615-9182-e9bfe8ed9877 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:47.948624 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e073df8-5a52-4615-9182-e9bfe8ed9877 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:47.992043 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd80be2-e18f-403c-b36a-192ab998c935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:47.992890 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd80be2-e18f-403c-b36a-192ab998c935 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:47.993182 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd80be2-e18f-403c-b36a-192ab998c935 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:47.993459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd80be2-e18f-403c-b36a-192ab998c935 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:48.002178 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd80be2-e18f-403c-b36a-192ab998c935 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:48.003652 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd80be2-e18f-403c-b36a-192ab998c935 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:48.018988 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-2fd80be2-e18f-403c-b36a-192ab998c935 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 12:19:48.018988 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:48.018988 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:48.018988 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:48.018988 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:48.018988 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:48.018988 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:48.018988 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:48.018988 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:48.018988 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:48.018988 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:48.018988 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:48.018988 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:19:48.018988 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:19:48.018988 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:19:48.018988 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:19:48.019996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:48.019996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:48.019996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:19:48.019996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:48.019996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:19:48.019996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:48.019996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:48.019996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:48.019996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:48.019996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:48.019996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:48.019996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:48.019996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:48.019996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:48.019996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:48.019996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:48.019996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:48.019996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:48.021222 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:48.021222 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:48.021222 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:19:48.021222 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:19:48.021222 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:19:48.021222 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:19:48.021222 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:19:48.021222 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:19:48.021222 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 12:19:48.021222 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:48.021222 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 379/760] 10.222.0.16 () {56 vars in 1093 bytes} [Mon May 23 12:19:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:48.026520 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa16ed28-2d9a-4533-806d-8b4be6967e21 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:48.176294 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa16ed28-2d9a-4533-806d-8b4be6967e21 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:48.201813 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929fc27f-cc7f-4baf-afa2-a0a12203e1a4 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:48.202253 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929fc27f-cc7f-4baf-afa2-a0a12203e1a4 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:48.202633 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929fc27f-cc7f-4baf-afa2-a0a12203e1a4 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:48.204343 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 380/761] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:48.210160 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ad59a-e1d2-4776-a221-1e139f9e0de0 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:48.210559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ad59a-e1d2-4776-a221-1e139f9e0de0 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:48.210916 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ad59a-e1d2-4776-a221-1e139f9e0de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:48.224662 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:48.232645 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-153ad59a-e1d2-4776-a221-1e139f9e0de0 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:48.233840 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-153ad59a-e1d2-4776-a221-1e139f9e0de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:48.236256 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3f79f-94cf-4f8f-a5bd-1bd41d267ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:48.236939 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3f79f-94cf-4f8f-a5bd-1bd41d267ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:48.237220 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3f79f-94cf-4f8f-a5bd-1bd41d267ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:48.237496 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af3f79f-94cf-4f8f-a5bd-1bd41d267ee2 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:48.238331 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af3f79f-94cf-4f8f-a5bd-1bd41d267ee2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:48.238642 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af3f79f-94cf-4f8f-a5bd-1bd41d267ee2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:48.241052 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af3f79f-94cf-4f8f-a5bd-1bd41d267ee2 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:48.278053 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 382/762] 10.222.0.16 () {56 vars in 1133 bytes} [Mon May 23 12:19:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:48.278962 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3f79f-94cf-4f8f-a5bd-1bd41d267ee2 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:48.352662 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:48.401719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 381/763] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 12:19:48.433545 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af3f79f-94cf-4f8f-a5bd-1bd41d267ee2 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:48.453246 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ad59a-e1d2-4776-a221-1e139f9e0de0 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:48.481238 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfe312f-f669-4606-96e4-d1dcbc84b672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:48.482802 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe312f-f669-4606-96e4-d1dcbc84b672 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:48.483316 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe312f-f669-4606-96e4-d1dcbc84b672 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:48.483805 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfe312f-f669-4606-96e4-d1dcbc84b672 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:48.495478 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfe312f-f669-4606-96e4-d1dcbc84b672 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:48.497647 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfe312f-f669-4606-96e4-d1dcbc84b672 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:48.522275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-4dfe312f-f669-4606-96e4-d1dcbc84b672 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:48.522275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:48.522275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:48.522275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:48.522275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:48.522275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:48.522275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:48.522275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:48.522275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:48.522275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:48.522275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:48.522275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:48.522275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:48.522275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:48.522275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:48.523260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:48.523260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:48.523260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:48.523260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:48.523260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:48.523260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:48.523260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:48.523260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:48.523260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:48.523260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:48.523260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:48.523260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:48.523260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:48.523260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:48.523260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:48.523260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:48.523260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:48.524187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:48.524187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:48.524187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:48.524187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:48.526496 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 383/764] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:48.532074 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfe312f-f669-4606-96e4-d1dcbc84b672 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:48.580641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:48.650358 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfe312f-f669-4606-96e4-d1dcbc84b672 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:48.690085 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ad59a-e1d2-4776-a221-1e139f9e0de0 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:48.691042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d4ca8-956b-4030-b16f-7195cefe4c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:48.692914 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d4ca8-956b-4030-b16f-7195cefe4c14 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:48.693665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d4ca8-956b-4030-b16f-7195cefe4c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:48.694188 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d4ca8-956b-4030-b16f-7195cefe4c14 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:48.695113 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d4ca8-956b-4030-b16f-7195cefe4c14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:48.695634 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d4ca8-956b-4030-b16f-7195cefe4c14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:48.698098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275d4ca8-956b-4030-b16f-7195cefe4c14 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:48.712525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 384/765] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:19:48.719237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d4ca8-956b-4030-b16f-7195cefe4c14 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:48.798281 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b929f-02fa-4e3f-a975-4097e812d99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:48.799145 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b929f-02fa-4e3f-a975-4097e812d99b None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:48.799816 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b929f-02fa-4e3f-a975-4097e812d99b None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:48.799816 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b929f-02fa-4e3f-a975-4097e812d99b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:48.800385 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b929f-02fa-4e3f-a975-4097e812d99b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:48.801809 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b929f-02fa-4e3f-a975-4097e812d99b None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:48.832799 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-e22b929f-02fa-4e3f-a975-4097e812d99b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 12:19:48.832799 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:48.832799 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 12:19:48.832799 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 12:19:48.832799 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 12:19:48.832799 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 12:19:48.832799 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 12:19:48.832799 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:19:48.832799 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 12:19:48.832799 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:48.832799 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:19:48.832799 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:48.832799 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:48.832799 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:48.832799 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:48.832799 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:48.834391 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:48.834391 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:48.834391 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:48.834391 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:48.834391 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:48.834391 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:48.834391 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:48.834391 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 12:19:48.834391 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 12:19:48.834391 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 12:19:48.834391 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 12:19:48.834391 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:48.834391 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:48.834391 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:48.834391 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:48.834391 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:48.835248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:48.835248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:48.835248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:48.835248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:48.835248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:48.835248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:48.835248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:48.835248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:48.835248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:48.835248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:48.835248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:48.835248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 12:19:48.835248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:19:48.835248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 12:19:48.835248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:48.840607 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 382/766] 10.222.0.16 () {56 vars in 1129 bytes} [Mon May 23 12:19:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:48.841374 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d4ca8-956b-4030-b16f-7195cefe4c14 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:48.850346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b929f-02fa-4e3f-a975-4097e812d99b None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:48.855484 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:48.928550 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:48.928874 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9880db1-73b4-4bba-a6c6-59ea2d9ba523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:48.929703 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9880db1-73b4-4bba-a6c6-59ea2d9ba523 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:48.929997 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9880db1-73b4-4bba-a6c6-59ea2d9ba523 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:48.930280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9880db1-73b4-4bba-a6c6-59ea2d9ba523 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:48.957093 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9880db1-73b4-4bba-a6c6-59ea2d9ba523 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:48.959053 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9880db1-73b4-4bba-a6c6-59ea2d9ba523 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:48.979817 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b929f-02fa-4e3f-a975-4097e812d99b None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:48.992477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-b9880db1-73b4-4bba-a6c6-59ea2d9ba523 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:48.992477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:48.992477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:48.992477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:48.992477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:48.992477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:48.992477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:48.992477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:48.992477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:48.992477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:48.992477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:48.992477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:48.992477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:19:48.992477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:19:48.992477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:19:48.993439 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:19:48.993439 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:48.993439 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:48.993439 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:48.993439 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:48.993439 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:48.993439 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:48.993439 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:48.993439 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:48.993439 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:48.993439 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:48.993439 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:48.993439 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:48.993439 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:48.993439 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:48.993439 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:48.993439 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:48.994336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:19:48.994336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:19:48.994336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:19:48.994336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:19:48.994336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:19:48.994336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:19:48.994336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:48.994336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:48.998413 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 385/767] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:48.999748 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35502f56-eb28-49f4-8538-ee0b22b50d70 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:49.000037 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35502f56-eb28-49f4-8538-ee0b22b50d70 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:49.000344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35502f56-eb28-49f4-8538-ee0b22b50d70 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:49.001863 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 386/768] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:49.002568 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35502f56-eb28-49f4-8538-ee0b22b50d70 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:49.033457 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07268e5e-4306-432b-85a6-b61449b76c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:49.034831 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07268e5e-4306-432b-85a6-b61449b76c1c None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:49.035291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07268e5e-4306-432b-85a6-b61449b76c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:49.035713 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07268e5e-4306-432b-85a6-b61449b76c1c None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:49.037393 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07268e5e-4306-432b-85a6-b61449b76c1c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:49.038160 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07268e5e-4306-432b-85a6-b61449b76c1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:49.040401 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07268e5e-4306-432b-85a6-b61449b76c1c None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:49.047420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 383/769] 10.222.0.16 () {56 vars in 1124 bytes} [Mon May 23 12:19:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:49.048874 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f7d79-8a39-44aa-9a98-e131253cf368 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:49.049293 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f7d79-8a39-44aa-9a98-e131253cf368 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:49.049697 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f7d79-8a39-44aa-9a98-e131253cf368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:49.070691 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-519f7d79-8a39-44aa-9a98-e131253cf368 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:49.072846 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-519f7d79-8a39-44aa-9a98-e131253cf368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:49.126216 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35502f56-eb28-49f4-8538-ee0b22b50d70 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:49.165506 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73995f5-4697-4d19-af84-2d453156d44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:49.165506 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73995f5-4697-4d19-af84-2d453156d44e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:49.165506 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73995f5-4697-4d19-af84-2d453156d44e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:49.165506 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73995f5-4697-4d19-af84-2d453156d44e None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:49.165955 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73995f5-4697-4d19-af84-2d453156d44e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:49.165955 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73995f5-4697-4d19-af84-2d453156d44e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:49.166098 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 384/770] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:19:49.168646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f7d79-8a39-44aa-9a98-e131253cf368 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:49.168771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73995f5-4697-4d19-af84-2d453156d44e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:49.182001 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 387/771] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:19:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:49.190616 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73995f5-4697-4d19-af84-2d453156d44e None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:49.228002 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:49.242794 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:49.313206 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f7d79-8a39-44aa-9a98-e131253cf368 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:49.334862 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73995f5-4697-4d19-af84-2d453156d44e None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:49.363714 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b1ec4-518e-4653-bc59-489daf882aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:49.364439 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b1ec4-518e-4653-bc59-489daf882aea None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:49.364712 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b1ec4-518e-4653-bc59-489daf882aea None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:49.364989 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b1ec4-518e-4653-bc59-489daf882aea None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:49.365711 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b1ec4-518e-4653-bc59-489daf882aea None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:49.366009 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b1ec4-518e-4653-bc59-489daf882aea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:49.367385 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b1ec4-518e-4653-bc59-489daf882aea None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:49.372707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 385/772] 10.222.0.16 () {56 vars in 1124 bytes} [Mon May 23 12:19:49 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:49.373599 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b1ec4-518e-4653-bc59-489daf882aea None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:49.382672 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1119f-a8d0-4621-863a-41a47c7d53f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:49.383963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1119f-a8d0-4621-863a-41a47c7d53f4 None admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:49.384497 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1119f-a8d0-4621-863a-41a47c7d53f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:49.384972 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1119f-a8d0-4621-863a-41a47c7d53f4 None admin] PATH_INFO: `/v3/projects/839585501f414e1ab02501422510e54f/users/b83535b443a84275a160cd84cb3ad5a0/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:49.387180 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef1119f-a8d0-4621-863a-41a47c7d53f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=839585501f414e1ab02501422510e54f, user_id=b83535b443a84275a160cd84cb3ad5a0, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:49.388855 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef1119f-a8d0-4621-863a-41a47c7d53f4 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:49.404722 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 388/773] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 12:19:49 2022] PUT /identity/v3/projects/839585501f414e1ab02501422510e54f/users/b83535b443a84275a160cd84cb3ad5a0/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:19:49.406162 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1119f-a8d0-4621-863a-41a47c7d53f4 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:49.489204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b1ec4-518e-4653-bc59-489daf882aea None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:49.502028 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:49.512185 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:49.528347 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb5990-5308-4b7f-a782-ab6fa96392df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:49.529152 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb5990-5308-4b7f-a782-ab6fa96392df None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:49.529438 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb5990-5308-4b7f-a782-ab6fa96392df None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:49.529728 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb5990-5308-4b7f-a782-ab6fa96392df None admin] PATH_INFO: `/v3/roles/service` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:49.540701 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1119f-a8d0-4621-863a-41a47c7d53f4 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:49.542141 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcb5990-5308-4b7f-a782-ab6fa96392df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:49.550014 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-0fcb5990-5308-4b7f-a782-ab6fa96392df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:49.550014 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:49.550014 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:49.550014 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:49.550014 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:49.550014 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:49.550014 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:49.550014 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:49.550014 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:49.550014 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:49.550014 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:49.550014 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:49.550014 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:49.550014 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:49.550014 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:49.550014 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:49.550985 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:49.550985 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:49.550985 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:49.550985 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:49.550985 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:49.550985 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:49.550985 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:49.550985 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:49.550985 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:49.550985 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:49.550985 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:49.550985 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:49.550985 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:49.550985 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:49.550985 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:49.550985 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:49.552909 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:49.552909 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:49.552909 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:49.552909 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:49.552909 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:49.552909 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:49.552909 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:49.552909 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:49.552909 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:49.552909 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:49.552909 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:49.552909 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:49.552909 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 386/774] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 12:19:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:49.557900 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb5990-5308-4b7f-a782-ab6fa96392df None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:49.584212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed8642-898b-4022-b144-8f32020ee42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:49.585360 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed8642-898b-4022-b144-8f32020ee42c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:49.585845 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed8642-898b-4022-b144-8f32020ee42c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:49.586431 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed8642-898b-4022-b144-8f32020ee42c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:49.587466 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed8642-898b-4022-b144-8f32020ee42c None admin] RBAC: Adding query filter params (interface=public, service_id=5344a213240f469083bc090a3c54e510, region_id=RegionOne) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:49.587995 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed8642-898b-4022-b144-8f32020ee42c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:49.589885 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed8642-898b-4022-b144-8f32020ee42c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:49.600498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 389/775] 10.222.0.16 () {56 vars in 1251 bytes} [Mon May 23 12:19:49 2022] GET /identity/v3/endpoints?service_id=5344a213240f469083bc090a3c54e510&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:49.607588 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed8642-898b-4022-b144-8f32020ee42c None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:49.673962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb5990-5308-4b7f-a782-ab6fa96392df None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:49.716631 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d54953-69eb-4779-8af8-b42579258d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:49.716858 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d54953-69eb-4779-8af8-b42579258d47 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:49.717092 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d54953-69eb-4779-8af8-b42579258d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:49.717302 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d54953-69eb-4779-8af8-b42579258d47 None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:49.718177 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d54953-69eb-4779-8af8-b42579258d47 None admin] RBAC: Adding query filter params (name=service) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:49.718490 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d54953-69eb-4779-8af8-b42579258d47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:49.720018 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d54953-69eb-4779-8af8-b42579258d47 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:49.730365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 387/776] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 12:19:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:49.734418 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed8642-898b-4022-b144-8f32020ee42c None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:49.736079 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d54953-69eb-4779-8af8-b42579258d47 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:49.770231 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:49.778561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf6ba6-4504-4ea4-b917-63a16b804040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:49.779481 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf6ba6-4504-4ea4-b917-63a16b804040 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:49.779910 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf6ba6-4504-4ea4-b917-63a16b804040 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:49.780609 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf6ba6-4504-4ea4-b917-63a16b804040 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:49.781570 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf6ba6-4504-4ea4-b917-63a16b804040 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:49.783157 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf6ba6-4504-4ea4-b917-63a16b804040 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:49.790976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 390/777] 10.222.0.16 () {56 vars in 1087 bytes} [Mon May 23 12:19:49 2022] GET /identity/v3/services => generated 772 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:49.854455 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d54953-69eb-4779-8af8-b42579258d47 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:49.894177 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4abc96-06b6-49d5-aad7-86de682a945f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:49.894983 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4abc96-06b6-49d5-aad7-86de682a945f None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:49.895268 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4abc96-06b6-49d5-aad7-86de682a945f None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:49.895605 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4abc96-06b6-49d5-aad7-86de682a945f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:49.908517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4abc96-06b6-49d5-aad7-86de682a945f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:49.910031 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4abc96-06b6-49d5-aad7-86de682a945f None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:49.912020 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee91941-d294-4b5c-9757-a0f89ea6106b None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:49.912698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee91941-d294-4b5c-9757-a0f89ea6106b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:49.913173 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee91941-d294-4b5c-9757-a0f89ea6106b None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:49.915825 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 391/778] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:19:49.920324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcceeaa-f5c5-43b1-94bf-3e5021320ada None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:49.921097 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcceeaa-f5c5-43b1-94bf-3e5021320ada None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:49.921577 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcceeaa-f5c5-43b1-94bf-3e5021320ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:49.929797 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-cd4abc96-06b6-49d5-aad7-86de682a945f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:49.929797 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:49.929797 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:49.929797 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:49.929797 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:49.929797 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:49.929797 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:49.929797 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:49.929797 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:49.929797 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:49.929797 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:49.929797 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:49.929797 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:49.929797 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:49.929797 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:49.930832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:49.930832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:49.930832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:49.930832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:49.930832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:49.930832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:49.930832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:49.930832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:49.930832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:49.930832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:49.930832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:49.930832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:49.930832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:49.930832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:49.930832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:49.930832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:49.930832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:49.932112 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:49.932112 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:49.932112 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:49.932112 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:49.932112 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 388/779] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:19:49.936982 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4abc96-06b6-49d5-aad7-86de682a945f None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:49.963038 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-0fcceeaa-f5c5-43b1-94bf-3e5021320ada None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:49.964764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-0fcceeaa-f5c5-43b1-94bf-3e5021320ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:49.967089 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:50.044921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:50.054783 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4abc96-06b6-49d5-aad7-86de682a945f None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:50.057379 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 392/780] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:19:50.058954 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5bc34-1d05-4e55-8fa9-5025a44d35ea None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:50.059245 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5bc34-1d05-4e55-8fa9-5025a44d35ea None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:50.059526 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5bc34-1d05-4e55-8fa9-5025a44d35ea None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:50.060857 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 393/781] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:50.068721 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d490a5a-936f-4c3a-ba6d-b971d4d9c340 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:50.069300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d490a5a-936f-4c3a-ba6d-b971d4d9c340 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:50.070011 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d490a5a-936f-4c3a-ba6d-b971d4d9c340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:50.092636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2e459-ee96-40a3-9aa7-1ace0ca9d6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:50.093354 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2e459-ee96-40a3-9aa7-1ace0ca9d6c8 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:50.093687 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2e459-ee96-40a3-9aa7-1ace0ca9d6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:50.093834 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-7d490a5a-936f-4c3a-ba6d-b971d4d9c340 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:50.093967 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2e459-ee96-40a3-9aa7-1ace0ca9d6c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:50.094704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d2e459-ee96-40a3-9aa7-1ace0ca9d6c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:50.095013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d2e459-ee96-40a3-9aa7-1ace0ca9d6c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:50.095603 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-7d490a5a-936f-4c3a-ba6d-b971d4d9c340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:50.096470 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d2e459-ee96-40a3-9aa7-1ace0ca9d6c8 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:50.109791 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 389/782] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:19:50.110868 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2e459-ee96-40a3-9aa7-1ace0ca9d6c8 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:50.188815 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 394/783] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:19:50.196193 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d490a5a-936f-4c3a-ba6d-b971d4d9c340 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:50.261940 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2e459-ee96-40a3-9aa7-1ace0ca9d6c8 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:50.286624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:50.312554 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5e44b-16b2-424f-9eea-3bde023f32ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:50.313895 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5e44b-16b2-424f-9eea-3bde023f32ea None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:50.314338 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5e44b-16b2-424f-9eea-3bde023f32ea None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:50.314992 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5e44b-16b2-424f-9eea-3bde023f32ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:50.325369 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d490a5a-936f-4c3a-ba6d-b971d4d9c340 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:50.334159 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd5e44b-16b2-424f-9eea-3bde023f32ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:50.344946 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-3cd5e44b-16b2-424f-9eea-3bde023f32ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:19:50.344946 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:50.344946 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:50.344946 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:50.344946 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:50.344946 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:50.344946 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:50.344946 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:50.344946 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:50.344946 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:50.344946 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:50.344946 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:50.344946 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:50.344946 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:50.344946 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:50.344946 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:50.346117 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:50.346117 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:50.346117 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:50.346117 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:50.346117 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:50.346117 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:50.346117 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:50.346117 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:50.346117 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:50.346117 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:50.346117 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:50.346117 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:50.346117 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:50.346117 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:50.346117 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:50.346117 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:50.347188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:50.347188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:50.347188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:50.347188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:50.347188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:50.347188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:50.347188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:50.347188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:50.347188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:50.347188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:50.347188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:19:50.347188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:50.349924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 390/784] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:19:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:50.351473 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5e44b-16b2-424f-9eea-3bde023f32ea None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:50.357512 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:50.370235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76828780-55cf-4a5a-aa94-d50954c645fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:50.371501 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76828780-55cf-4a5a-aa94-d50954c645fe None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:50.372096 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76828780-55cf-4a5a-aa94-d50954c645fe None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:50.372698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76828780-55cf-4a5a-aa94-d50954c645fe None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:50.382199 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76828780-55cf-4a5a-aa94-d50954c645fe None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:50.383968 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76828780-55cf-4a5a-aa94-d50954c645fe None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:50.401168 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-76828780-55cf-4a5a-aa94-d50954c645fe None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 12:19:50.401168 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:50.401168 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:50.401168 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:50.401168 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:50.401168 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:50.401168 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:50.401168 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:50.401168 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:50.401168 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:50.401168 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:50.401168 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:50.401168 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:19:50.401168 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:19:50.401168 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:19:50.401168 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:19:50.402177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:50.402177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:50.402177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:19:50.402177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:50.402177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:19:50.402177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:50.402177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:50.402177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:50.402177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:50.402177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:50.402177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:50.402177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:50.402177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:50.402177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:50.402177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:50.402177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:50.402177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:50.402177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:50.403120 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:50.403120 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:50.403120 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:19:50.403120 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:19:50.403120 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:19:50.403120 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:19:50.403120 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:19:50.403120 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:19:50.403120 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 12:19:50.403120 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:50.405873 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 395/785] 10.222.0.16 () {56 vars in 1117 bytes} [Mon May 23 12:19:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:50.406696 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76828780-55cf-4a5a-aa94-d50954c645fe None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:50.481904 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5e44b-16b2-424f-9eea-3bde023f32ea None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:50.530110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2f9f0-f3f1-4dbe-bd8f-6612c2ee92fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:50.531558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2f9f0-f3f1-4dbe-bd8f-6612c2ee92fa None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:50.532313 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2f9f0-f3f1-4dbe-bd8f-6612c2ee92fa None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:50.533245 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2f9f0-f3f1-4dbe-bd8f-6612c2ee92fa None admin] PATH_INFO: `/v3/services/network` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:50.534380 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b2f9f0-f3f1-4dbe-bd8f-6612c2ee92fa None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:50.537871 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b2f9f0-f3f1-4dbe-bd8f-6612c2ee92fa None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:50.542359 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76828780-55cf-4a5a-aa94-d50954c645fe None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:50.554853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-c5b2f9f0-f3f1-4dbe-bd8f-6612c2ee92fa None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 23 12:19:50.554853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:50.554853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 12:19:50.554853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 12:19:50.554853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 12:19:50.554853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 12:19:50.554853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 12:19:50.554853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:19:50.554853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 12:19:50.554853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:50.554853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:19:50.554853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:50.554853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:50.554853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:50.554853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:50.554853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:50.554853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:50.555880 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:50.555880 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:50.555880 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:50.555880 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:50.555880 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:50.555880 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:50.555880 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 12:19:50.555880 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 12:19:50.555880 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 12:19:50.555880 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 12:19:50.555880 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:50.555880 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:50.555880 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:50.555880 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:50.555880 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:50.555880 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:50.556762 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:50.556762 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:50.556762 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:50.556762 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:50.556762 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:50.556762 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:50.556762 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:50.556762 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:50.556762 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:50.556762 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:50.556762 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 12:19:50.556762 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:19:50.556762 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 12:19:50.556762 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:50.563141 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 391/786] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:50 2022] GET /identity/v3/services/network => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:50.563141 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2f9f0-f3f1-4dbe-bd8f-6612c2ee92fa None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:50.581440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:50.594898 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36823c77-3f2e-48a1-b526-4f242f54f9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:50.596082 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36823c77-3f2e-48a1-b526-4f242f54f9e2 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:50.596828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36823c77-3f2e-48a1-b526-4f242f54f9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:50.597701 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36823c77-3f2e-48a1-b526-4f242f54f9e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:50.599715 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36823c77-3f2e-48a1-b526-4f242f54f9e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:50.600328 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36823c77-3f2e-48a1-b526-4f242f54f9e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:50.603071 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36823c77-3f2e-48a1-b526-4f242f54f9e2 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:50.614676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 396/787] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:19:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:50.616121 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36823c77-3f2e-48a1-b526-4f242f54f9e2 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:50.636520 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:50.674604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2f9f0-f3f1-4dbe-bd8f-6612c2ee92fa None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:50.720697 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f9d075-fb01-43f4-885f-795ae5f1ee0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:50.721769 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9d075-fb01-43f4-885f-795ae5f1ee0b None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:50.722788 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9d075-fb01-43f4-885f-795ae5f1ee0b None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:50.723279 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9d075-fb01-43f4-885f-795ae5f1ee0b None admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:50.725193 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f9d075-fb01-43f4-885f-795ae5f1ee0b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:50.725797 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f9d075-fb01-43f4-885f-795ae5f1ee0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:50.728095 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f9d075-fb01-43f4-885f-795ae5f1ee0b None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:50.748880 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 392/788] 10.222.0.16 () {56 vars in 1149 bytes} [Mon May 23 12:19:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:50.749397 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36823c77-3f2e-48a1-b526-4f242f54f9e2 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:50.750223 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f9d075-fb01-43f4-885f-795ae5f1ee0b None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:50.787259 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c907590-3152-4cf5-bf01-52bb4a59b61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:50.788101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c907590-3152-4cf5-bf01-52bb4a59b61f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:50.788453 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c907590-3152-4cf5-bf01-52bb4a59b61f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:50.789333 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c907590-3152-4cf5-bf01-52bb4a59b61f None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:50.789574 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c907590-3152-4cf5-bf01-52bb4a59b61f None admin] RBAC: Adding query filter params (name=network) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:50.789956 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c907590-3152-4cf5-bf01-52bb4a59b61f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:50.791394 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c907590-3152-4cf5-bf01-52bb4a59b61f None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:50.800815 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 397/789] 10.222.0.16 () {56 vars in 1112 bytes} [Mon May 23 12:19:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:50.800815 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c907590-3152-4cf5-bf01-52bb4a59b61f None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:50.862323 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:50.868211 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f9d075-fb01-43f4-885f-795ae5f1ee0b None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:50.896604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:50.905679 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362661b-9536-4e38-b0b9-51b89b3f725f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:50.906942 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362661b-9536-4e38-b0b9-51b89b3f725f None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:50.907853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362661b-9536-4e38-b0b9-51b89b3f725f None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:50.908750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362661b-9536-4e38-b0b9-51b89b3f725f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:50.910322 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c907590-3152-4cf5-bf01-52bb4a59b61f None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:50.922953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d362661b-9536-4e38-b0b9-51b89b3f725f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:50.924911 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d362661b-9536-4e38-b0b9-51b89b3f725f None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:50.947379 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db771e-b7ed-4e96-a5eb-d4d67f5f140e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:50.948200 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db771e-b7ed-4e96-a5eb-d4d67f5f140e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:50.948491 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db771e-b7ed-4e96-a5eb-d4d67f5f140e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:50.948771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db771e-b7ed-4e96-a5eb-d4d67f5f140e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:50.949408 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-d362661b-9536-4e38-b0b9-51b89b3f725f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:50.949408 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:50.949408 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:50.949408 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:50.949408 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:50.949408 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:50.949408 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:50.949408 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:50.949408 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:50.949408 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:50.949408 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:50.949408 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:50.949408 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:50.949408 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:50.949408 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:50.950292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:50.950292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:50.950292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:50.950292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:50.950292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:50.950292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:50.950292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:50.950292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:50.950292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:50.950292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:50.950292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:50.950292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:50.950292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:50.950292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:50.950292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:50.950292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:50.950292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:50.951197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:50.951197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:50.951197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:50.951197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:50.953581 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 393/790] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:50.954611 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362661b-9536-4e38-b0b9-51b89b3f725f None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:50.967162 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6db771e-b7ed-4e96-a5eb-d4d67f5f140e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:50.969558 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6db771e-b7ed-4e96-a5eb-d4d67f5f140e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:50.993755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-b6db771e-b7ed-4e96-a5eb-d4d67f5f140e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:50.993755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:50.993755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:50.993755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:50.993755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:50.993755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:50.993755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:50.993755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:50.993755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:50.993755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:50.993755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:50.993755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:50.993755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:50.993755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:50.993755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:50.994677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:50.994677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:50.994677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:50.994677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:50.994677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:50.994677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:50.994677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:50.994677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:50.994677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:50.994677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:50.994677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:50.994677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:50.994677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:50.994677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:50.994677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:50.994677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:50.994677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:50.995581 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:50.995581 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:50.995581 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:50.995581 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:50.997803 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 398/791] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:50.999178 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db771e-b7ed-4e96-a5eb-d4d67f5f140e None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:51.067894 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362661b-9536-4e38-b0b9-51b89b3f725f None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:51.114093 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8090e-3484-4e60-9b68-274d30707d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:51.115554 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8090e-3484-4e60-9b68-274d30707d54 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:51.115941 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8090e-3484-4e60-9b68-274d30707d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:51.116348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8090e-3484-4e60-9b68-274d30707d54 None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:51.117403 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8090e-3484-4e60-9b68-274d30707d54 None admin] RBAC: Adding query filter params (type=network) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:51.117812 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8090e-3484-4e60-9b68-274d30707d54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:51.119683 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a8090e-3484-4e60-9b68-274d30707d54 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:51.128251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 394/792] 10.222.0.16 () {56 vars in 1112 bytes} [Mon May 23 12:19:50 2022] GET /identity/v3/services?type=network => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:51.129249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8090e-3484-4e60-9b68-274d30707d54 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:51.144847 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db771e-b7ed-4e96-a5eb-d4d67f5f140e None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:51.155004 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:51.182599 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:51.194009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97e85c-4ef0-4c8b-a67f-b0b246abe127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:51.194788 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97e85c-4ef0-4c8b-a67f-b0b246abe127 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:51.195167 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97e85c-4ef0-4c8b-a67f-b0b246abe127 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:51.195538 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97e85c-4ef0-4c8b-a67f-b0b246abe127 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:51.196747 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af97e85c-4ef0-4c8b-a67f-b0b246abe127 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:51.197069 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af97e85c-4ef0-4c8b-a67f-b0b246abe127 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:51.198437 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af97e85c-4ef0-4c8b-a67f-b0b246abe127 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:51.215526 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 399/793] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:51.216857 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97e85c-4ef0-4c8b-a67f-b0b246abe127 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:51.251458 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a8090e-3484-4e60-9b68-274d30707d54 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:51.293451 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b08d25-4de5-45c2-b233-5fa163d91076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:51.294503 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b08d25-4de5-45c2-b233-5fa163d91076 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:51.295236 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b08d25-4de5-45c2-b233-5fa163d91076 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:51.296083 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b08d25-4de5-45c2-b233-5fa163d91076 None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:51.297288 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b08d25-4de5-45c2-b233-5fa163d91076 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:51.297789 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b08d25-4de5-45c2-b233-5fa163d91076 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:51.300288 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b08d25-4de5-45c2-b233-5fa163d91076 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:51.316643 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 395/794] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:51.318660 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b08d25-4de5-45c2-b233-5fa163d91076 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:51.339406 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97e85c-4ef0-4c8b-a67f-b0b246abe127 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:51.375446 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a9e47-ecd9-432f-8e5e-32205f0aaae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:51.375989 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a9e47-ecd9-432f-8e5e-32205f0aaae7 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:51.376398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a9e47-ecd9-432f-8e5e-32205f0aaae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:51.376751 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a9e47-ecd9-432f-8e5e-32205f0aaae7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:51.377627 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00a9e47-ecd9-432f-8e5e-32205f0aaae7 None admin] RBAC: Adding query filter params (interface=public, service_id=13800a6c24b3487b8f08d2471442bb63, region_id=RegionOne) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:51.378012 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00a9e47-ecd9-432f-8e5e-32205f0aaae7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:51.379512 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00a9e47-ecd9-432f-8e5e-32205f0aaae7 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:51.390442 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 400/795] 10.222.0.16 () {56 vars in 1251 bytes} [Mon May 23 12:19:51 2022] GET /identity/v3/endpoints?service_id=13800a6c24b3487b8f08d2471442bb63&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:51.391449 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a9e47-ecd9-432f-8e5e-32205f0aaae7 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:51.431038 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:51.450977 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b08d25-4de5-45c2-b233-5fa163d91076 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:51.452826 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:51.492967 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095c5cb-7b91-4a81-b5dc-863c98cfcd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:51.493888 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095c5cb-7b91-4a81-b5dc-863c98cfcd40 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:51.494346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095c5cb-7b91-4a81-b5dc-863c98cfcd40 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:51.494581 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d095c5cb-7b91-4a81-b5dc-863c98cfcd40 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:51.503768 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095c5cb-7b91-4a81-b5dc-863c98cfcd40 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:51.505770 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d095c5cb-7b91-4a81-b5dc-863c98cfcd40 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:51.522018 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-d095c5cb-7b91-4a81-b5dc-863c98cfcd40 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 12:19:51.522018 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:51.522018 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:51.522018 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:51.522018 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:51.522018 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:51.522018 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:51.522018 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:51.522018 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:51.522018 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:51.522018 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:51.522018 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:51.522018 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:19:51.522018 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:19:51.522018 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:19:51.522018 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:19:51.523001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:51.523001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:51.523001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:19:51.523001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:51.523001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:19:51.523001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:51.523001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:51.523001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:51.523001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:51.523001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:51.523001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:51.523001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:51.523001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:51.523001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:51.523001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:51.523001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:51.523001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:51.523001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:51.523943 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:51.523943 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:51.523943 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:19:51.523943 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:19:51.523943 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:19:51.523943 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:19:51.523943 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:19:51.523943 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:19:51.523943 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 12:19:51.523943 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:51.523943 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 396/796] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:19:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:51.524569 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095c5cb-7b91-4a81-b5dc-863c98cfcd40 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:51.536942 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a9e47-ecd9-432f-8e5e-32205f0aaae7 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:51.583888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da46880-2a80-4711-a270-fb1e5595ca2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:51.585926 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da46880-2a80-4711-a270-fb1e5595ca2a None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:51.586358 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da46880-2a80-4711-a270-fb1e5595ca2a None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:51.587026 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da46880-2a80-4711-a270-fb1e5595ca2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:51.595396 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da46880-2a80-4711-a270-fb1e5595ca2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:51.597067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da46880-2a80-4711-a270-fb1e5595ca2a None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:51.612484 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-7da46880-2a80-4711-a270-fb1e5595ca2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:51.612484 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:51.612484 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:51.612484 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:51.612484 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:51.612484 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:51.612484 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:51.612484 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:51.612484 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:51.612484 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:51.612484 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:51.612484 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:51.612484 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:19:51.612484 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:19:51.612484 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:19:51.613602 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:19:51.613602 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:51.613602 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:51.613602 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:51.613602 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:51.613602 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:51.613602 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:51.613602 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:51.613602 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:51.613602 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:51.613602 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:51.613602 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:51.613602 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:51.613602 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:51.613602 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:51.613602 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:51.613602 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:51.615334 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:19:51.615334 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:19:51.615334 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:19:51.615334 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:19:51.615334 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:19:51.615334 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:19:51.615334 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:51.615334 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:51.616579 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 401/797] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:51.617452 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da46880-2a80-4711-a270-fb1e5595ca2a None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:51.663226 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d095c5cb-7b91-4a81-b5dc-863c98cfcd40 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:51.727122 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10031068-98e7-4cee-9fd6-051fbe793b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:51.728023 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10031068-98e7-4cee-9fd6-051fbe793b2d None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:51.728455 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10031068-98e7-4cee-9fd6-051fbe793b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:51.728843 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10031068-98e7-4cee-9fd6-051fbe793b2d None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:51.729549 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10031068-98e7-4cee-9fd6-051fbe793b2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:51.730942 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10031068-98e7-4cee-9fd6-051fbe793b2d None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:51.750172 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 397/798] 10.222.0.16 () {56 vars in 1087 bytes} [Mon May 23 12:19:51 2022] GET /identity/v3/services => generated 772 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:51.750539 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10031068-98e7-4cee-9fd6-051fbe793b2d None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:51.847897 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:51.857669 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:51.873145 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da46880-2a80-4711-a270-fb1e5595ca2a None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:52.028845 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10031068-98e7-4cee-9fd6-051fbe793b2d None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:52.041142 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7eb9d5-a9d3-407b-adb4-30216e79a932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:52.041853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7eb9d5-a9d3-407b-adb4-30216e79a932 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:52.042143 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7eb9d5-a9d3-407b-adb4-30216e79a932 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:52.042430 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7eb9d5-a9d3-407b-adb4-30216e79a932 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:52.043293 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7eb9d5-a9d3-407b-adb4-30216e79a932 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:52.043612 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7eb9d5-a9d3-407b-adb4-30216e79a932 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:52.049545 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7eb9d5-a9d3-407b-adb4-30216e79a932 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:52.070196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d798c7b-00ab-4b53-b98f-20f56c9223af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:52.071246 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d798c7b-00ab-4b53-b98f-20f56c9223af None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:52.071536 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d798c7b-00ab-4b53-b98f-20f56c9223af None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:52.071819 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d798c7b-00ab-4b53-b98f-20f56c9223af None admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:52.072761 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d798c7b-00ab-4b53-b98f-20f56c9223af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:52.073083 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d798c7b-00ab-4b53-b98f-20f56c9223af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:52.074563 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d798c7b-00ab-4b53-b98f-20f56c9223af None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:52.095409 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 398/799] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:19:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:52.096770 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11cc62-035d-4618-92e6-0cec1bc1b36b None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:52.097066 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11cc62-035d-4618-92e6-0cec1bc1b36b None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:52.097278 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 402/800] 10.222.0.16 () {56 vars in 1135 bytes} [Mon May 23 12:19:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 480 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:52.098323 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11cc62-035d-4618-92e6-0cec1bc1b36b None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:52.098966 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7eb9d5-a9d3-407b-adb4-30216e79a932 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:52.104477 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 399/801] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:52.104477 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11cc62-035d-4618-92e6-0cec1bc1b36b None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:52.214687 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:52.219297 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7eb9d5-a9d3-407b-adb4-30216e79a932 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:52.232672 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11cc62-035d-4618-92e6-0cec1bc1b36b None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:52.258804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5415e7d-784b-4891-817f-64ad0752a6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:52.259948 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5415e7d-784b-4891-817f-64ad0752a6de None admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:52.260262 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5415e7d-784b-4891-817f-64ad0752a6de None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:52.260548 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5415e7d-784b-4891-817f-64ad0752a6de None admin] PATH_INFO: `/v3/projects/839585501f414e1ab02501422510e54f/users/733f8296ec3d4795a1140c1e1cb6eb95/roles/81d82471e2be4f0aa8164b75c16a9718` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:52.268885 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65e3a4-0d9d-487d-bbd8-7b9545077812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:52.269669 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65e3a4-0d9d-487d-bbd8-7b9545077812 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:52.269952 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65e3a4-0d9d-487d-bbd8-7b9545077812 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:52.270230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65e3a4-0d9d-487d-bbd8-7b9545077812 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:52.275711 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5415e7d-784b-4891-817f-64ad0752a6de None admin] RBAC: Authorizing `identity:create_grant(project_id=839585501f414e1ab02501422510e54f, user_id=733f8296ec3d4795a1140c1e1cb6eb95, role_id=81d82471e2be4f0aa8164b75c16a9718)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:52.277294 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5415e7d-784b-4891-817f-64ad0752a6de None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:52.281166 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65e3a4-0d9d-487d-bbd8-7b9545077812 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:52.282874 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65e3a4-0d9d-487d-bbd8-7b9545077812 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:52.286927 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:52.293069 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 403/802] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 12:19:52 2022] PUT /identity/v3/projects/839585501f414e1ab02501422510e54f/users/733f8296ec3d4795a1140c1e1cb6eb95/roles/81d82471e2be4f0aa8164b75c16a9718 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:19:52.294219 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202e2d7-e766-47bd-aafb-1b2953412f57 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:52.295110 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202e2d7-e766-47bd-aafb-1b2953412f57 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:52.295414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202e2d7-e766-47bd-aafb-1b2953412f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:52.314827 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-8202e2d7-e766-47bd-aafb-1b2953412f57 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:52.316215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-8202e2d7-e766-47bd-aafb-1b2953412f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:52.319721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-dd65e3a4-0d9d-487d-bbd8-7b9545077812 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:52.319721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:52.319721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:52.319721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:52.319721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:52.319721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:52.319721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:52.319721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:52.319721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:52.319721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:52.319721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:52.319721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:52.319721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:52.319721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:52.319721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:52.320917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:52.320917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:52.320917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:52.320917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:52.320917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:52.320917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:52.320917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:52.320917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:52.320917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:52.320917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:52.320917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:52.320917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:52.320917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:52.320917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:52.320917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:52.320917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:52.320917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:52.322081 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:52.322081 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:52.322081 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:52.322081 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:52.325759 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 400/803] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:52.328557 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4792a76-8f2b-49fd-873d-27ce50c22faf None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:52.330252 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4792a76-8f2b-49fd-873d-27ce50c22faf None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:52.331529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4792a76-8f2b-49fd-873d-27ce50c22faf None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:52.335363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 401/804] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:52.336546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4792a76-8f2b-49fd-873d-27ce50c22faf None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:52.408700 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 404/805] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:19:52.410632 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2565bcf-02d1-4837-b577-bd6cb8123a8b None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:52.411239 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2565bcf-02d1-4837-b577-bd6cb8123a8b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:52.411741 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2565bcf-02d1-4837-b577-bd6cb8123a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:52.431541 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a2565bcf-02d1-4837-b577-bd6cb8123a8b None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:52.433195 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a2565bcf-02d1-4837-b577-bd6cb8123a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:52.489501 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4792a76-8f2b-49fd-873d-27ce50c22faf None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:52.518057 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 405/806] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:19:52.518998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2565bcf-02d1-4837-b577-bd6cb8123a8b None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:52.550677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e1614-c1d8-4aaa-8c1a-1c04d5ca2da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:52.552465 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e1614-c1d8-4aaa-8c1a-1c04d5ca2da5 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:52.552739 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e1614-c1d8-4aaa-8c1a-1c04d5ca2da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:52.553301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7e1614-c1d8-4aaa-8c1a-1c04d5ca2da5 None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:52.554301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e1614-c1d8-4aaa-8c1a-1c04d5ca2da5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:52.554622 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e1614-c1d8-4aaa-8c1a-1c04d5ca2da5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:52.557518 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7e1614-c1d8-4aaa-8c1a-1c04d5ca2da5 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:52.565926 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:52.573335 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 402/807] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:52.574278 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e1614-c1d8-4aaa-8c1a-1c04d5ca2da5 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:52.600594 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:52.641343 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2565bcf-02d1-4837-b577-bd6cb8123a8b None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:52.688960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24134a3e-ba24-4e42-ac87-5a0e565dd498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:52.690453 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24134a3e-ba24-4e42-ac87-5a0e565dd498 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:52.690843 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24134a3e-ba24-4e42-ac87-5a0e565dd498 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:52.691443 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24134a3e-ba24-4e42-ac87-5a0e565dd498 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:52.694370 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7e1614-c1d8-4aaa-8c1a-1c04d5ca2da5 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:52.705810 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24134a3e-ba24-4e42-ac87-5a0e565dd498 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:52.717289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-24134a3e-ba24-4e42-ac87-5a0e565dd498 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:19:52.717289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:52.717289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:52.717289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:52.717289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:52.717289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:52.717289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:52.717289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:52.717289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:52.717289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:52.717289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:52.717289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:52.717289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:52.717289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:52.717289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:52.717289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:52.718297 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:52.718297 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:52.718297 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:52.718297 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:52.718297 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:52.718297 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:52.718297 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:52.718297 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:52.718297 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:52.718297 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:52.718297 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:52.718297 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:52.718297 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:52.718297 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:52.718297 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:52.718297 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:52.719166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:52.719166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:52.719166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:52.719166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:52.719166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:52.719166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:52.719166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:52.719166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:52.719166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:52.719166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:52.719166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:19:52.719166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:52.720313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 406/808] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:19:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:52.721556 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24134a3e-ba24-4e42-ac87-5a0e565dd498 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:52.737924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9b31f-1a6e-4efe-ac24-328a8a11116e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:52.738745 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9b31f-1a6e-4efe-ac24-328a8a11116e None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:52.739051 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9b31f-1a6e-4efe-ac24-328a8a11116e None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:52.739750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9b31f-1a6e-4efe-ac24-328a8a11116e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:52.740453 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9b31f-1a6e-4efe-ac24-328a8a11116e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:52.741825 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9b31f-1a6e-4efe-ac24-328a8a11116e None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:52.754338 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-87f9b31f-1a6e-4efe-ac24-328a8a11116e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 12:19:52.754338 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:52.754338 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 12:19:52.754338 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 12:19:52.754338 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 12:19:52.754338 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 12:19:52.754338 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 12:19:52.754338 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:19:52.754338 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 12:19:52.754338 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:52.754338 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:19:52.754338 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:52.754338 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:52.754338 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:52.754338 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:52.754338 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:52.755883 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:52.755883 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:52.755883 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:52.755883 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:52.755883 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:52.755883 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:52.755883 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:52.755883 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 12:19:52.755883 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 12:19:52.755883 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 12:19:52.755883 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 12:19:52.755883 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:52.755883 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:52.755883 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:52.755883 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:52.755883 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:52.756779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:52.756779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:52.756779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:52.756779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:52.756779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:52.756779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:52.756779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:52.756779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:52.756779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:52.756779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:52.756779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:52.756779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 12:19:52.756779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:19:52.756779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 12:19:52.756779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:52.756779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 403/809] 10.222.0.16 () {56 vars in 1129 bytes} [Mon May 23 12:19:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:52.758312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9b31f-1a6e-4efe-ac24-328a8a11116e None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:52.827593 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:52.857136 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24134a3e-ba24-4e42-ac87-5a0e565dd498 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:52.876384 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9b31f-1a6e-4efe-ac24-328a8a11116e None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:52.880362 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:52.905437 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1c3a6-5059-45b0-a8f3-409136dbf3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:52.907071 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1c3a6-5059-45b0-a8f3-409136dbf3c2 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:52.907583 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1c3a6-5059-45b0-a8f3-409136dbf3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:52.908222 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1c3a6-5059-45b0-a8f3-409136dbf3c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:52.914569 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd58af8-40ff-4ff1-8294-1494d1bb35cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:52.915314 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd58af8-40ff-4ff1-8294-1494d1bb35cd None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:52.915671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd58af8-40ff-4ff1-8294-1494d1bb35cd None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:52.916019 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd58af8-40ff-4ff1-8294-1494d1bb35cd None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:52.916961 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd58af8-40ff-4ff1-8294-1494d1bb35cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:52.917225 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd58af8-40ff-4ff1-8294-1494d1bb35cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:52.918315 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c1c3a6-5059-45b0-a8f3-409136dbf3c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:52.918751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd58af8-40ff-4ff1-8294-1494d1bb35cd None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:52.920381 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c1c3a6-5059-45b0-a8f3-409136dbf3c2 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:52.928334 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 404/810] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:19:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:52.932809 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd58af8-40ff-4ff1-8294-1494d1bb35cd None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:52.938880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-c7c1c3a6-5059-45b0-a8f3-409136dbf3c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:52.938880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:52.938880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:52.938880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:52.938880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:52.938880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:52.938880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:52.938880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:52.938880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:52.938880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:52.938880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:52.938880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:52.938880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:19:52.938880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:19:52.938880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:19:52.939835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:19:52.939835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:52.939835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:52.939835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:52.939835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:52.939835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:52.939835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:52.939835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:52.939835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:52.939835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:52.939835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:52.939835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:52.939835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:52.939835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:52.939835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:52.939835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:52.939835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:52.940803 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:19:52.940803 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:19:52.940803 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:19:52.940803 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:19:52.940803 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:19:52.940803 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:19:52.940803 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:52.940803 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:52.943730 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 407/811] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:52.945256 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1c3a6-5059-45b0-a8f3-409136dbf3c2 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:53.046748 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd58af8-40ff-4ff1-8294-1494d1bb35cd None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:53.061673 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1c3a6-5059-45b0-a8f3-409136dbf3c2 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:53.073910 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:53.087986 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff388e-8881-429c-92e4-02e3ac339fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:53.088692 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff388e-8881-429c-92e4-02e3ac339fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:53.089025 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff388e-8881-429c-92e4-02e3ac339fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:53.089464 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff388e-8881-429c-92e4-02e3ac339fb8 None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:53.090116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff388e-8881-429c-92e4-02e3ac339fb8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:53.090425 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff388e-8881-429c-92e4-02e3ac339fb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:53.091935 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff388e-8881-429c-92e4-02e3ac339fb8 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:53.101793 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 405/812] 10.222.0.16 () {56 vars in 1124 bytes} [Mon May 23 12:19:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:53.102337 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a44caa5-bfb1-42bf-8dbc-881c204eaabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:53.102668 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff388e-8881-429c-92e4-02e3ac339fb8 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:53.103695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44caa5-bfb1-42bf-8dbc-881c204eaabf None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:53.104235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44caa5-bfb1-42bf-8dbc-881c204eaabf None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:53.104738 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a44caa5-bfb1-42bf-8dbc-881c204eaabf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:53.118122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a44caa5-bfb1-42bf-8dbc-881c204eaabf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:53.119990 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a44caa5-bfb1-42bf-8dbc-881c204eaabf None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:53.140552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-1a44caa5-bfb1-42bf-8dbc-881c204eaabf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:53.140552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:53.140552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:53.140552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:53.140552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:53.140552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:53.140552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:53.140552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:53.140552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:53.140552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:53.140552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:53.140552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:53.140552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:53.140552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:53.140552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:53.141536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:53.141536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:53.141536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:53.141536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:53.141536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:53.141536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:53.141536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:53.141536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:53.141536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:53.141536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:53.141536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:53.141536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:53.141536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:53.141536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:53.141536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:53.141536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:53.141536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:53.142433 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:53.142433 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:53.142433 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:53.142433 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:53.144431 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 408/813] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:53.145534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a44caa5-bfb1-42bf-8dbc-881c204eaabf None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:53.176431 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:53.223248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff388e-8881-429c-92e4-02e3ac339fb8 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:53.251377 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a44caa5-bfb1-42bf-8dbc-881c204eaabf None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:53.261133 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d62da0-0cf6-4523-8a66-65aa1d13ef35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:53.261804 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d62da0-0cf6-4523-8a66-65aa1d13ef35 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:53.262092 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d62da0-0cf6-4523-8a66-65aa1d13ef35 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:53.262374 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d62da0-0cf6-4523-8a66-65aa1d13ef35 None admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:53.263172 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d62da0-0cf6-4523-8a66-65aa1d13ef35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:53.263486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d62da0-0cf6-4523-8a66-65aa1d13ef35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:53.265056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d62da0-0cf6-4523-8a66-65aa1d13ef35 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:53.279159 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 406/814] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:19:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:53.279884 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d62da0-0cf6-4523-8a66-65aa1d13ef35 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:53.290657 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f34af27-644a-4dc6-95e0-a962cae73eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:53.291621 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34af27-644a-4dc6-95e0-a962cae73eac None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:53.292111 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34af27-644a-4dc6-95e0-a962cae73eac None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:53.292661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f34af27-644a-4dc6-95e0-a962cae73eac None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:53.293573 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f34af27-644a-4dc6-95e0-a962cae73eac None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:53.294109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f34af27-644a-4dc6-95e0-a962cae73eac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:53.295675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f34af27-644a-4dc6-95e0-a962cae73eac None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:53.301085 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 409/815] 10.222.0.16 () {56 vars in 1124 bytes} [Mon May 23 12:19:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:53.302604 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f34af27-644a-4dc6-95e0-a962cae73eac None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:53.339634 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:53.398792 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:53.405133 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d62da0-0cf6-4523-8a66-65aa1d13ef35 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:53.406261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f34af27-644a-4dc6-95e0-a962cae73eac None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:53.443266 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b415d-abe8-40ed-8320-c3be329f9eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:53.444325 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1fc78b-8590-414a-8820-032fe1e0aca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:53.444893 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b415d-abe8-40ed-8320-c3be329f9eb7 None admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:53.445002 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1fc78b-8590-414a-8820-032fe1e0aca2 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:53.445271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1fc78b-8590-414a-8820-032fe1e0aca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:53.445552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1fc78b-8590-414a-8820-032fe1e0aca2 None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:53.445897 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b415d-abe8-40ed-8320-c3be329f9eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:53.446276 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1fc78b-8590-414a-8820-032fe1e0aca2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:53.446592 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1fc78b-8590-414a-8820-032fe1e0aca2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:53.446694 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b415d-abe8-40ed-8320-c3be329f9eb7 None admin] PATH_INFO: `/v3/projects/839585501f414e1ab02501422510e54f/users/71fb9a79ac494635804b93d4a0048205/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:53.447969 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1fc78b-8590-414a-8820-032fe1e0aca2 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:53.449227 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b415d-abe8-40ed-8320-c3be329f9eb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=839585501f414e1ab02501422510e54f, user_id=71fb9a79ac494635804b93d4a0048205, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:53.450875 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2b415d-abe8-40ed-8320-c3be329f9eb7 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:53.463802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 407/816] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:53.463802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1fc78b-8590-414a-8820-032fe1e0aca2 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:53.480339 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 410/817] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 12:19:53 2022] PUT /identity/v3/projects/839585501f414e1ab02501422510e54f/users/71fb9a79ac494635804b93d4a0048205/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:19:53.481229 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b415d-abe8-40ed-8320-c3be329f9eb7 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:53.668722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1fc78b-8590-414a-8820-032fe1e0aca2 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:53.684927 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:53.690081 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b415d-abe8-40ed-8320-c3be329f9eb7 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:53.744448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4228e-bb35-4b40-ad14-5db640d9ad8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:53.745041 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4228e-bb35-4b40-ad14-5db640d9ad8d None admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:53.745325 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4228e-bb35-4b40-ad14-5db640d9ad8d None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:53.745604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4228e-bb35-4b40-ad14-5db640d9ad8d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:53.746461 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4228e-bb35-4b40-ad14-5db640d9ad8d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:53.747969 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4228e-bb35-4b40-ad14-5db640d9ad8d None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:53.752677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea68462-e52d-4676-a60b-4e0d941a6bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:53.753532 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea68462-e52d-4676-a60b-4e0d941a6bca None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:53.753822 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea68462-e52d-4676-a60b-4e0d941a6bca None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:53.754102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea68462-e52d-4676-a60b-4e0d941a6bca None admin] PATH_INFO: `/v3/users/nova` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:53.768208 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea68462-e52d-4676-a60b-4e0d941a6bca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:53.772516 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea68462-e52d-4676-a60b-4e0d941a6bca None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:53.777583 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 408/818] 10.222.0.16 () {60 vars in 1144 bytes} [Mon May 23 12:19:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:19:53.796280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-6ea68462-e52d-4676-a60b-4e0d941a6bca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 12:19:53.796280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:53.796280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:53.796280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:53.796280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:53.796280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:53.796280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:53.796280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:53.796280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:53.796280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:53.796280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:53.796280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:53.796280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:19:53.796280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:19:53.796280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:19:53.796280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:19:53.797287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:53.797287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:53.797287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:19:53.797287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:53.797287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:19:53.797287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:53.797287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:53.797287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:53.797287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:53.797287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:53.797287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:53.797287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:53.797287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:53.797287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:53.797287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:53.797287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:53.797287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:53.797287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:53.798257 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:53.798257 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:53.798257 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:19:53.798257 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:19:53.798257 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:19:53.798257 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:19:53.798257 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:19:53.798257 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:19:53.798257 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 12:19:53.798257 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:53.798257 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 411/819] 10.222.0.16 () {56 vars in 1093 bytes} [Mon May 23 12:19:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:53.804381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4228e-bb35-4b40-ad14-5db640d9ad8d None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:53.934247 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4228e-bb35-4b40-ad14-5db640d9ad8d None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:53.976740 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab5efc-4c3a-462c-a850-2e3e6fafc7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:53.977481 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab5efc-4c3a-462c-a850-2e3e6fafc7b9 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:53.977768 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab5efc-4c3a-462c-a850-2e3e6fafc7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:53.978048 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab5efc-4c3a-462c-a850-2e3e6fafc7b9 None admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:53.978951 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fab5efc-4c3a-462c-a850-2e3e6fafc7b9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:53.979272 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fab5efc-4c3a-462c-a850-2e3e6fafc7b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:53.980744 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fab5efc-4c3a-462c-a850-2e3e6fafc7b9 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:54.006251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 409/820] 10.222.0.16 () {56 vars in 1133 bytes} [Mon May 23 12:19:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:19:54.011637 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea68462-e52d-4676-a60b-4e0d941a6bca None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:54.033554 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:54.142003 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea68462-e52d-4676-a60b-4e0d941a6bca None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:54.186027 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc464d3e-02c1-4fb6-b42f-e20d994eafff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:54.186709 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc464d3e-02c1-4fb6-b42f-e20d994eafff None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:54.186994 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc464d3e-02c1-4fb6-b42f-e20d994eafff None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:54.187268 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc464d3e-02c1-4fb6-b42f-e20d994eafff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:54.263261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc464d3e-02c1-4fb6-b42f-e20d994eafff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:54.264751 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc464d3e-02c1-4fb6-b42f-e20d994eafff None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:54.280236 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:54.374545 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 412/821] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:19:54 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 12:19:54.377356 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab5efc-4c3a-462c-a850-2e3e6fafc7b9 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:54.387441 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bea940-70e4-498b-9e3f-33bd2a8b7465 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:54.387823 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bea940-70e4-498b-9e3f-33bd2a8b7465 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:54.388112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bea940-70e4-498b-9e3f-33bd2a8b7465 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:54.389788 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 413/822] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:19:54.397839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed62301-6764-4042-8f20-59e9c328e054 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:54.398107 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed62301-6764-4042-8f20-59e9c328e054 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:54.398391 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed62301-6764-4042-8f20-59e9c328e054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:54.409745 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:54.429624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-eed62301-6764-4042-8f20-59e9c328e054 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:54.430823 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-eed62301-6764-4042-8f20-59e9c328e054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:54.515609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab5efc-4c3a-462c-a850-2e3e6fafc7b9 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:54.568786 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650baa8-212c-4ef3-aaa9-7807199e71b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:54.569598 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650baa8-212c-4ef3-aaa9-7807199e71b7 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:54.569875 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650baa8-212c-4ef3-aaa9-7807199e71b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:54.570146 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650baa8-212c-4ef3-aaa9-7807199e71b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:54.589479 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650baa8-212c-4ef3-aaa9-7807199e71b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:54.590945 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650baa8-212c-4ef3-aaa9-7807199e71b7 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:54.603234 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 414/823] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:19:54.617187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-6650baa8-212c-4ef3-aaa9-7807199e71b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:54.617187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:54.617187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:54.617187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:54.617187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:54.617187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:54.617187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:54.617187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:54.617187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:54.617187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:54.617187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:54.617187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:54.617187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:54.617187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:54.617187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:54.619219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:54.619219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:54.619219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:54.619219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:54.619219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:54.619219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:54.619219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:54.619219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:54.619219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:54.619219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:54.619219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:54.619219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:54.619219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:54.619219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:54.619219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:54.619219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:54.619219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:54.621361 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:54.621361 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:54.621361 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:54.621361 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:54.624191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 410/824] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:54.624377 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed62301-6764-4042-8f20-59e9c328e054 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:54.632696 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650baa8-212c-4ef3-aaa9-7807199e71b7 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:54.692907 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:54.766815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:54.777188 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed62301-6764-4042-8f20-59e9c328e054 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:54.809551 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650baa8-212c-4ef3-aaa9-7807199e71b7 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:54.823935 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9e676-6ada-402e-8fae-9c516b9c8f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:54.825083 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9e676-6ada-402e-8fae-9c516b9c8f40 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:54.825399 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9e676-6ada-402e-8fae-9c516b9c8f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:54.825684 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9e676-6ada-402e-8fae-9c516b9c8f40 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:54.826722 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9e676-6ada-402e-8fae-9c516b9c8f40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:54.827036 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9e676-6ada-402e-8fae-9c516b9c8f40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:54.828706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9e676-6ada-402e-8fae-9c516b9c8f40 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:54.845328 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 415/825] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:54.847394 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701ad7a-c162-4723-9dfa-fb0ec882d3aa None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:54.847695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701ad7a-c162-4723-9dfa-fb0ec882d3aa None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:54.847987 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701ad7a-c162-4723-9dfa-fb0ec882d3aa None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:54.853744 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 416/826] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:54.853744 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701ad7a-c162-4723-9dfa-fb0ec882d3aa None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:54.866721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf23a6-910d-4a3a-8d50-8b34b4683e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:54.867961 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf23a6-910d-4a3a-8d50-8b34b4683e46 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:54.869078 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf23a6-910d-4a3a-8d50-8b34b4683e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:54.869568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf23a6-910d-4a3a-8d50-8b34b4683e46 None admin] PATH_INFO: `/v3/services/network` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:54.871007 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf23a6-910d-4a3a-8d50-8b34b4683e46 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:54.873354 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf23a6-910d-4a3a-8d50-8b34b4683e46 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:54.891308 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-8adf23a6-910d-4a3a-8d50-8b34b4683e46 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 23 12:19:54.891308 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:54.891308 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 12:19:54.891308 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 12:19:54.891308 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 12:19:54.891308 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 12:19:54.891308 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 12:19:54.891308 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:19:54.891308 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 12:19:54.891308 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:54.891308 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:19:54.891308 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:54.891308 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:54.891308 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:54.891308 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:54.891308 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:54.891308 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:54.892684 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:54.892684 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:54.892684 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:54.892684 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:54.892684 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:54.892684 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:54.892684 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 12:19:54.892684 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 12:19:54.892684 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 12:19:54.892684 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 12:19:54.892684 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:54.892684 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:54.892684 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:54.892684 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:54.892684 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:54.892684 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:54.893531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:54.893531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:54.893531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:54.893531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:54.893531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:54.893531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:54.893531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:54.893531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:54.893531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:54.893531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:54.893531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 12:19:54.893531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:19:54.893531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 12:19:54.893531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:54.897133 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 411/827] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:54 2022] GET /identity/v3/services/network => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:19:54.901937 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a463bd-6998-4da6-8c2c-2a3dc37dc553 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:54.902518 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a463bd-6998-4da6-8c2c-2a3dc37dc553 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:54.903004 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a463bd-6998-4da6-8c2c-2a3dc37dc553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:54.925484 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-60a463bd-6998-4da6-8c2c-2a3dc37dc553 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:54.928501 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-60a463bd-6998-4da6-8c2c-2a3dc37dc553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:54.983024 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701ad7a-c162-4723-9dfa-fb0ec882d3aa None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:54.986692 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:55.012200 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 412/828] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:19:55.012200 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a463bd-6998-4da6-8c2c-2a3dc37dc553 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:55.053168 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd35a29-4a1b-40ec-b5f5-d49da70c82b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:55.053958 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd35a29-4a1b-40ec-b5f5-d49da70c82b4 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:55.054237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd35a29-4a1b-40ec-b5f5-d49da70c82b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:55.054508 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd35a29-4a1b-40ec-b5f5-d49da70c82b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:55.066918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd35a29-4a1b-40ec-b5f5-d49da70c82b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:55.069445 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd35a29-4a1b-40ec-b5f5-d49da70c82b4 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:55.090363 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-2bd35a29-4a1b-40ec-b5f5-d49da70c82b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:55.090363 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:55.090363 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:55.090363 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:55.090363 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:55.090363 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:55.090363 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:55.090363 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:55.090363 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:55.090363 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:55.090363 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:55.090363 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:55.090363 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:19:55.090363 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:19:55.090363 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:19:55.091367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:19:55.091367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:55.091367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:55.091367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:55.091367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:55.091367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:55.091367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:55.091367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:55.091367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:55.091367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:55.091367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:55.091367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:55.091367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:55.091367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:55.091367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:55.091367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:55.091367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:55.092319 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:19:55.092319 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:19:55.092319 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:19:55.092319 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:19:55.092319 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:19:55.092319 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:19:55.092319 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:55.092319 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:55.099655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 417/829] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:55.099655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd35a29-4a1b-40ec-b5f5-d49da70c82b4 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:55.105675 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:55.147103 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a463bd-6998-4da6-8c2c-2a3dc37dc553 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:55.194043 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a36407-2992-46b6-9269-9371f341172a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:55.195316 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a36407-2992-46b6-9269-9371f341172a None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:55.195810 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a36407-2992-46b6-9269-9371f341172a None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:55.196556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a36407-2992-46b6-9269-9371f341172a None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:55.198464 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a36407-2992-46b6-9269-9371f341172a None admin] RBAC: Adding query filter params (name=network) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:55.199013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a36407-2992-46b6-9269-9371f341172a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:55.200633 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a36407-2992-46b6-9269-9371f341172a None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:55.208485 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 413/830] 10.222.0.16 () {56 vars in 1112 bytes} [Mon May 23 12:19:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:55.210116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a36407-2992-46b6-9269-9371f341172a None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:55.228720 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd35a29-4a1b-40ec-b5f5-d49da70c82b4 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:55.272800 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f663f-c8b0-41ec-ae49-d3f99611b54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:55.274621 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f663f-c8b0-41ec-ae49-d3f99611b54a None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:55.275099 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f663f-c8b0-41ec-ae49-d3f99611b54a None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:55.275567 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f663f-c8b0-41ec-ae49-d3f99611b54a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:55.291024 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f663f-c8b0-41ec-ae49-d3f99611b54a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:55.301576 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-d32f663f-c8b0-41ec-ae49-d3f99611b54a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:55.301576 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:55.301576 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:55.301576 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:55.301576 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:55.301576 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:55.301576 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:55.301576 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:55.301576 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:55.301576 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:55.301576 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:55.301576 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:55.301576 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:55.301576 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:55.301576 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:55.301576 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:55.302534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:55.302534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:55.302534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:55.302534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:55.302534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:55.302534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:55.302534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:55.302534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:55.302534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:55.302534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:55.302534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:55.302534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:55.302534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:55.302534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:55.302534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:55.302534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:55.303407 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:55.303407 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:55.303407 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:55.303407 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:55.303407 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:55.303407 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:55.303407 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:55.303407 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:55.303407 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:55.303407 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:55.303407 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 12:19:55.303407 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:55.311267 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 418/831] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 12:19:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:55.311267 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f663f-c8b0-41ec-ae49-d3f99611b54a None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:55.339081 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:55.347786 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a36407-2992-46b6-9269-9371f341172a None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:55.386111 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:55.400090 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66292abf-f55d-4532-a0df-aaef3a1c8244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:55.401184 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66292abf-f55d-4532-a0df-aaef3a1c8244 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:55.401701 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66292abf-f55d-4532-a0df-aaef3a1c8244 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:55.402304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66292abf-f55d-4532-a0df-aaef3a1c8244 None admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:55.403465 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66292abf-f55d-4532-a0df-aaef3a1c8244 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:55.404029 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66292abf-f55d-4532-a0df-aaef3a1c8244 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:55.405755 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66292abf-f55d-4532-a0df-aaef3a1c8244 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:55.423599 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f663f-c8b0-41ec-ae49-d3f99611b54a None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:55.424202 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 414/832] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:19:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:55.425410 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66292abf-f55d-4532-a0df-aaef3a1c8244 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:55.462690 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10287a4c-08f0-4d83-a1fb-23eba14143a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:55.463353 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10287a4c-08f0-4d83-a1fb-23eba14143a2 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:55.463680 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10287a4c-08f0-4d83-a1fb-23eba14143a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:55.463963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10287a4c-08f0-4d83-a1fb-23eba14143a2 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:55.464882 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10287a4c-08f0-4d83-a1fb-23eba14143a2 None admin] RBAC: Adding query filter params (type=network) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:55.465196 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10287a4c-08f0-4d83-a1fb-23eba14143a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:55.466724 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10287a4c-08f0-4d83-a1fb-23eba14143a2 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:55.472778 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 419/833] 10.222.0.16 () {56 vars in 1112 bytes} [Mon May 23 12:19:55 2022] GET /identity/v3/services?type=network => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:55.474065 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10287a4c-08f0-4d83-a1fb-23eba14143a2 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:55.540960 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66292abf-f55d-4532-a0df-aaef3a1c8244 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:55.571020 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:55.580643 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e45d2-d509-460f-87b9-f13359c6c0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:55.581681 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e45d2-d509-460f-87b9-f13359c6c0a4 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:55.582265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e45d2-d509-460f-87b9-f13359c6c0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:55.583377 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e45d2-d509-460f-87b9-f13359c6c0a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:55.584523 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e45d2-d509-460f-87b9-f13359c6c0a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:55.585042 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e45d2-d509-460f-87b9-f13359c6c0a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:55.585370 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10287a4c-08f0-4d83-a1fb-23eba14143a2 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:55.586961 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e45d2-d509-460f-87b9-f13359c6c0a4 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:55.600578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 415/834] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 12:19:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:55.601725 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e45d2-d509-460f-87b9-f13359c6c0a4 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:55.622694 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef477751-3424-44cb-8401-90465b3d92e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:55.623356 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef477751-3424-44cb-8401-90465b3d92e6 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:55.623636 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef477751-3424-44cb-8401-90465b3d92e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:55.623966 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef477751-3424-44cb-8401-90465b3d92e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:55.624984 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef477751-3424-44cb-8401-90465b3d92e6 None admin] RBAC: Adding query filter params (role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8, scope.project.id=839585501f414e1ab02501422510e54f, user.id=b83535b443a84275a160cd84cb3ad5a0) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:55.625294 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef477751-3424-44cb-8401-90465b3d92e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:55.626752 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef477751-3424-44cb-8401-90465b3d92e6 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:55.640201 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 420/835] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:19:55 2022] GET /identity/v3/role_assignments?user.id=b83535b443a84275a160cd84cb3ad5a0&scope.project.id=839585501f414e1ab02501422510e54f&role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:55.640201 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef477751-3424-44cb-8401-90465b3d92e6 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:55.675204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:55.732769 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e45d2-d509-460f-87b9-f13359c6c0a4 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:55.767509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef477751-3424-44cb-8401-90465b3d92e6 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:55.772803 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec10156-16d1-4dda-b3cc-4501475f0e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:55.773844 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec10156-16d1-4dda-b3cc-4501475f0e48 None admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:55.774344 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec10156-16d1-4dda-b3cc-4501475f0e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:55.774832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec10156-16d1-4dda-b3cc-4501475f0e48 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:55.775886 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec10156-16d1-4dda-b3cc-4501475f0e48 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:55.777497 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec10156-16d1-4dda-b3cc-4501475f0e48 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:55.803125 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 416/836] 10.222.0.16 () {60 vars in 1144 bytes} [Mon May 23 12:19:55 2022] POST /identity/v3/endpoints => generated 327 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:19:55.815189 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd67646-b7f5-487f-8f1b-dbf894dbb0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:55.816122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd67646-b7f5-487f-8f1b-dbf894dbb0e7 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:55.816444 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd67646-b7f5-487f-8f1b-dbf894dbb0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:55.816775 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd67646-b7f5-487f-8f1b-dbf894dbb0e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:55.832343 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd67646-b7f5-487f-8f1b-dbf894dbb0e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:55.835187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd67646-b7f5-487f-8f1b-dbf894dbb0e7 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:55.846909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:55.857735 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-0bd67646-b7f5-487f-8f1b-dbf894dbb0e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:55.857735 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:55.857735 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:55.857735 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:55.857735 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:55.857735 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:55.857735 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:55.857735 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:55.857735 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:55.857735 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:55.857735 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:55.857735 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:55.857735 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:55.857735 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:55.857735 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:55.858776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:55.858776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:55.858776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:55.858776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:55.858776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:55.858776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:55.858776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:55.858776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:55.858776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:55.858776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:55.858776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:55.858776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:55.858776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:55.858776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:55.858776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:55.858776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:55.858776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:55.859685 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:55.859685 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:55.859685 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:55.859685 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:55.865963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 421/837] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:55.872399 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec10156-16d1-4dda-b3cc-4501475f0e48 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:55.993390 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec10156-16d1-4dda-b3cc-4501475f0e48 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:56.002232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:56.038646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3f3da-31f8-4d68-9d72-9da0470cd22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:56.039342 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3f3da-31f8-4d68-9d72-9da0470cd22c None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:56.039632 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3f3da-31f8-4d68-9d72-9da0470cd22c None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:56.039921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3f3da-31f8-4d68-9d72-9da0470cd22c None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:56.040710 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca3f3da-31f8-4d68-9d72-9da0470cd22c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:56.041025 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca3f3da-31f8-4d68-9d72-9da0470cd22c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:56.042452 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca3f3da-31f8-4d68-9d72-9da0470cd22c None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:56.046695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb64f67-e88c-4006-a7e6-327a603d5319 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:56.047273 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb64f67-e88c-4006-a7e6-327a603d5319 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:56.047560 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb64f67-e88c-4006-a7e6-327a603d5319 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:56.049323 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 422/838] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:56.056317 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8dd41-7983-4f02-9b08-fd92a5076e71 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:56.056609 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8dd41-7983-4f02-9b08-fd92a5076e71 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:56.056890 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8dd41-7983-4f02-9b08-fd92a5076e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:56.060268 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 417/839] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:56.061517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3f3da-31f8-4d68-9d72-9da0470cd22c None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:56.076724 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-b2d8dd41-7983-4f02-9b08-fd92a5076e71 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:56.077912 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b2d8dd41-7983-4f02-9b08-fd92a5076e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:56.160297 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 423/840] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 12:19:56.173837 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3f3da-31f8-4d68-9d72-9da0470cd22c None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:56.182733 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8dd41-7983-4f02-9b08-fd92a5076e71 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:56.210727 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621e2a4-06a1-4946-b1d7-2a691f4756a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:56.211552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621e2a4-06a1-4946-b1d7-2a691f4756a8 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:56.211838 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621e2a4-06a1-4946-b1d7-2a691f4756a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:56.212120 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621e2a4-06a1-4946-b1d7-2a691f4756a8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:56.220658 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621e2a4-06a1-4946-b1d7-2a691f4756a8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:56.222140 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621e2a4-06a1-4946-b1d7-2a691f4756a8 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:56.239232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-d621e2a4-06a1-4946-b1d7-2a691f4756a8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 12:19:56.239232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:56.239232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:56.239232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:56.239232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:56.239232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:56.239232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:56.239232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:56.239232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:56.239232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:56.239232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:56.239232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:56.239232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:19:56.239232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:19:56.239232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:19:56.239232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:19:56.240449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:56.240449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:56.240449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:19:56.240449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:56.240449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:19:56.240449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:56.240449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:56.240449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:56.240449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:56.240449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:56.240449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:56.240449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:56.240449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:56.240449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:56.240449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:56.240449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:56.240449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:56.240449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:56.241405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:56.241405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:56.241405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:19:56.241405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:19:56.241405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:19:56.241405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:19:56.241405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:19:56.241405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:19:56.241405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 12:19:56.241405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:56.241405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 418/841] 10.222.0.16 () {56 vars in 1117 bytes} [Mon May 23 12:19:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:19:56.246628 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621e2a4-06a1-4946-b1d7-2a691f4756a8 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:56.277216 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:56.295074 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8dd41-7983-4f02-9b08-fd92a5076e71 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:56.327223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:56.334833 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5645fc-52a3-45ea-9740-da4a3c4364f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:56.335825 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5645fc-52a3-45ea-9740-da4a3c4364f3 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:56.336471 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5645fc-52a3-45ea-9740-da4a3c4364f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:56.336897 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5645fc-52a3-45ea-9740-da4a3c4364f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:56.349002 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5645fc-52a3-45ea-9740-da4a3c4364f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:19:56.357775 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-ba5645fc-52a3-45ea-9740-da4a3c4364f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:19:56.357775 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:56.357775 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:56.357775 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:56.357775 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:56.357775 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:56.357775 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:56.357775 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:56.357775 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:56.357775 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:56.357775 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:56.357775 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:56.357775 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:19:56.357775 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:19:56.357775 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:19:56.357775 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:19:56.359641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:19:56.359641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:19:56.359641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:19:56.359641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:19:56.359641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:56.359641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:56.359641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:56.359641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:56.359641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:56.359641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:56.359641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:56.359641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:56.359641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:56.359641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:56.359641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:56.359641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:56.361662 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:56.361662 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:56.361662 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:56.361662 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:56.361662 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:19:56.361662 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:19:56.361662 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:19:56.361662 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:19:56.361662 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:19:56.361662 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:19:56.361662 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:19:56.361662 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:56.363157 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621e2a4-06a1-4946-b1d7-2a691f4756a8 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:56.372196 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 424/842] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:19:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:56.372196 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5645fc-52a3-45ea-9740-da4a3c4364f3 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:56.397560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe100d-ce56-47ee-a14a-3c699db42196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:56.398349 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe100d-ce56-47ee-a14a-3c699db42196 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:56.398644 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe100d-ce56-47ee-a14a-3c699db42196 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:56.398982 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe100d-ce56-47ee-a14a-3c699db42196 None admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:56.399829 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe100d-ce56-47ee-a14a-3c699db42196 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:56.400291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe100d-ce56-47ee-a14a-3c699db42196 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:56.401856 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe100d-ce56-47ee-a14a-3c699db42196 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:56.426052 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 419/843] 10.222.0.16 () {56 vars in 1149 bytes} [Mon May 23 12:19:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:19:56.427460 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe100d-ce56-47ee-a14a-3c699db42196 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:56.478066 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5645fc-52a3-45ea-9740-da4a3c4364f3 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:56.515520 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ffa45-662b-4031-962a-598be6822404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:56.516328 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ffa45-662b-4031-962a-598be6822404 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:56.516613 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ffa45-662b-4031-962a-598be6822404 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:56.516889 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ffa45-662b-4031-962a-598be6822404 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:56.517640 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ffa45-662b-4031-962a-598be6822404 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:56.517946 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ffa45-662b-4031-962a-598be6822404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:56.519551 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ffa45-662b-4031-962a-598be6822404 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:56.522263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:56.538185 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 425/844] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:19:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:19:56.539680 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ffa45-662b-4031-962a-598be6822404 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:56.544544 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe100d-ce56-47ee-a14a-3c699db42196 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:56.584380 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4025dc2-53c9-4a6d-8370-b40bdd5625b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:56.585039 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4025dc2-53c9-4a6d-8370-b40bdd5625b4 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:56.585327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4025dc2-53c9-4a6d-8370-b40bdd5625b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:56.585611 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4025dc2-53c9-4a6d-8370-b40bdd5625b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:56.603166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:56.659901 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ffa45-662b-4031-962a-598be6822404 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:56.663979 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4025dc2-53c9-4a6d-8370-b40bdd5625b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:56.665985 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4025dc2-53c9-4a6d-8370-b40bdd5625b4 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:56.698358 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b45ca-c2c4-48c5-a01e-24dc8b0208e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:56.699173 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b45ca-c2c4-48c5-a01e-24dc8b0208e4 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:56.699465 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b45ca-c2c4-48c5-a01e-24dc8b0208e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:56.699740 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b45ca-c2c4-48c5-a01e-24dc8b0208e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:56.711916 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725b45ca-c2c4-48c5-a01e-24dc8b0208e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:56.713555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725b45ca-c2c4-48c5-a01e-24dc8b0208e4 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:56.732998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-725b45ca-c2c4-48c5-a01e-24dc8b0208e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:56.732998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:56.732998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:56.732998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:56.732998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:56.732998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:56.732998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:56.732998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:56.732998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:56.732998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:56.732998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:56.732998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:56.732998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:56.732998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:56.732998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:56.734166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:56.734166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:56.734166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:56.734166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:56.734166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:56.734166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:56.734166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:56.734166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:56.734166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:56.734166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:56.734166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:56.734166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:56.734166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:56.734166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:56.734166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:56.734166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:56.734166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:56.735193 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:56.735193 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:56.735193 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:56.735193 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:56.738831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 426/845] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:19:56.741370 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29191096-9be0-42ed-8a71-38ef1edb1c06 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:56.742116 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29191096-9be0-42ed-8a71-38ef1edb1c06 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:56.742116 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29191096-9be0-42ed-8a71-38ef1edb1c06 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:56.743429 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 427/846] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:56.743955 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29191096-9be0-42ed-8a71-38ef1edb1c06 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:56.762400 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:56.782606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 420/847] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:19:56 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 12:19:56.784110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4567b7f-4c1c-4ee5-ae74-a649fd6202ea None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:56.784560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4567b7f-4c1c-4ee5-ae74-a649fd6202ea None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:56.785005 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4567b7f-4c1c-4ee5-ae74-a649fd6202ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:56.804482 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-e4567b7f-4c1c-4ee5-ae74-a649fd6202ea None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:56.805930 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-e4567b7f-4c1c-4ee5-ae74-a649fd6202ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:56.855130 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29191096-9be0-42ed-8a71-38ef1edb1c06 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:56.859144 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:56.878823 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 421/848] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:19:56.879441 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4567b7f-4c1c-4ee5-ae74-a649fd6202ea None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:56.907244 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b4d50-078e-4275-8be0-30897a5e61bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:56.908982 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b4d50-078e-4275-8be0-30897a5e61bf None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:56.909462 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b4d50-078e-4275-8be0-30897a5e61bf None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:56.910170 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b4d50-078e-4275-8be0-30897a5e61bf None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:56.912330 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b4d50-078e-4275-8be0-30897a5e61bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:56.912829 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b4d50-078e-4275-8be0-30897a5e61bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:56.914637 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b4d50-078e-4275-8be0-30897a5e61bf None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:56.929670 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 428/849] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:56.930554 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b4d50-078e-4275-8be0-30897a5e61bf None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:56.986314 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4567b7f-4c1c-4ee5-ae74-a649fd6202ea None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:57.001419 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:57.031697 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8200225-87df-4dd8-b55e-b5a6e5dd82c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:57.032626 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8200225-87df-4dd8-b55e-b5a6e5dd82c7 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:57.032989 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8200225-87df-4dd8-b55e-b5a6e5dd82c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:57.033342 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8200225-87df-4dd8-b55e-b5a6e5dd82c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:57.044591 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8200225-87df-4dd8-b55e-b5a6e5dd82c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:57.046006 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8200225-87df-4dd8-b55e-b5a6e5dd82c7 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:57.054191 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b4d50-078e-4275-8be0-30897a5e61bf None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:57.065508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-f8200225-87df-4dd8-b55e-b5a6e5dd82c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:57.065508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:57.065508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:57.065508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:57.065508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:57.065508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:57.065508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:57.065508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:57.065508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:57.065508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:57.065508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:57.065508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:57.065508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:57.065508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:57.065508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:57.066405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:57.066405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:57.066405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:57.066405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:57.066405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:57.066405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:57.066405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:57.066405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:57.066405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:57.066405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:57.066405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:57.066405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:57.066405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:57.066405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:57.066405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:57.066405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:57.066405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:57.067587 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:57.067587 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:57.067587 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:57.067587 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:57.067587 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 422/850] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:57.068567 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8200225-87df-4dd8-b55e-b5a6e5dd82c7 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:57.101039 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d0d39-dc93-41ae-b993-4bd6bcb9057d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:57.102168 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d0d39-dc93-41ae-b993-4bd6bcb9057d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:57.102719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d0d39-dc93-41ae-b993-4bd6bcb9057d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:57.103009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2d0d39-dc93-41ae-b993-4bd6bcb9057d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:57.103674 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2d0d39-dc93-41ae-b993-4bd6bcb9057d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:57.106008 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2d0d39-dc93-41ae-b993-4bd6bcb9057d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:57.121424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-6b2d0d39-dc93-41ae-b993-4bd6bcb9057d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 12:19:57.121424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:57.121424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 12:19:57.121424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 12:19:57.121424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 12:19:57.121424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 12:19:57.121424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 12:19:57.121424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:19:57.121424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 12:19:57.121424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:57.121424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:19:57.121424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:57.121424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:57.121424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:57.121424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:57.121424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:57.121424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:57.122456 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:57.122456 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:57.122456 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:57.122456 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:57.122456 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:57.122456 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:57.122456 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 12:19:57.122456 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 12:19:57.122456 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 12:19:57.122456 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 12:19:57.122456 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:57.122456 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:57.122456 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:57.122456 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:57.122456 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:57.122456 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:57.128992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:57.128992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:57.128992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:57.128992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:57.128992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:57.128992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:57.128992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:57.128992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:57.128992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:57.128992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:57.128992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 12:19:57.128992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:19:57.128992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 12:19:57.128992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:57.128992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 429/851] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 12:19:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:57.130373 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d0d39-dc93-41ae-b993-4bd6bcb9057d None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:57.157754 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:57.174354 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8200225-87df-4dd8-b55e-b5a6e5dd82c7 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:57.214362 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c9171-b2da-4260-9a8e-a96a047ddee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:57.215160 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c9171-b2da-4260-9a8e-a96a047ddee1 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:57.215446 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c9171-b2da-4260-9a8e-a96a047ddee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:57.215732 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c9171-b2da-4260-9a8e-a96a047ddee1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:57.224426 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c9171-b2da-4260-9a8e-a96a047ddee1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:57.225886 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814c9171-b2da-4260-9a8e-a96a047ddee1 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:57.234615 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2d0d39-dc93-41ae-b993-4bd6bcb9057d None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:57.241863 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-814c9171-b2da-4260-9a8e-a96a047ddee1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 12:19:57.241863 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:57.241863 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:57.241863 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:57.241863 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:57.241863 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:57.241863 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:57.241863 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:57.241863 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:57.241863 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:57.241863 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:57.241863 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:57.241863 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:19:57.241863 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:19:57.241863 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:19:57.241863 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:19:57.242805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:57.242805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:57.242805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:19:57.242805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:57.242805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:19:57.242805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:19:57.242805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:57.242805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:57.242805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:57.242805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:57.242805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:57.242805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:57.242805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:57.242805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:57.242805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:57.242805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:57.242805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:57.242805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:57.243780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:57.243780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:57.243780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:19:57.243780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:19:57.243780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:19:57.243780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:19:57.243780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:19:57.243780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:19:57.243780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 12:19:57.243780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:57.243780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 423/852] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:19:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:57.243780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c9171-b2da-4260-9a8e-a96a047ddee1 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:57.272671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:57.275065 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb35c3-c736-4526-b2ed-7a7bf00ab0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:57.275865 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb35c3-c736-4526-b2ed-7a7bf00ab0f9 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:57.276205 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb35c3-c736-4526-b2ed-7a7bf00ab0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:57.276464 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb35c3-c736-4526-b2ed-7a7bf00ab0f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:57.277343 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb35c3-c736-4526-b2ed-7a7bf00ab0f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:57.277518 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb35c3-c736-4526-b2ed-7a7bf00ab0f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:57.278828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb35c3-c736-4526-b2ed-7a7bf00ab0f9 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:57.295962 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 430/853] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:57.295962 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb35c3-c736-4526-b2ed-7a7bf00ab0f9 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:57.355281 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c9171-b2da-4260-9a8e-a96a047ddee1 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:57.389091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:57.390707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b423b88-abb5-45a0-bc61-024b06e68936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:57.391489 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b423b88-abb5-45a0-bc61-024b06e68936 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:57.391771 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b423b88-abb5-45a0-bc61-024b06e68936 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:57.392104 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b423b88-abb5-45a0-bc61-024b06e68936 None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:57.392832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b423b88-abb5-45a0-bc61-024b06e68936 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:57.393178 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b423b88-abb5-45a0-bc61-024b06e68936 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:57.394536 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b423b88-abb5-45a0-bc61-024b06e68936 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:57.399577 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 424/854] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 12:19:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:57.403111 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb35c3-c736-4526-b2ed-7a7bf00ab0f9 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:57.404766 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b423b88-abb5-45a0-bc61-024b06e68936 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:57.442543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350ad55-ba49-4491-b8cd-38709546235a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:57.443555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350ad55-ba49-4491-b8cd-38709546235a None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:57.444046 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350ad55-ba49-4491-b8cd-38709546235a None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:57.444565 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1350ad55-ba49-4491-b8cd-38709546235a None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:57.445599 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1350ad55-ba49-4491-b8cd-38709546235a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:57.446511 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1350ad55-ba49-4491-b8cd-38709546235a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:57.448500 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1350ad55-ba49-4491-b8cd-38709546235a None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:57.467840 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 431/855] 10.222.0.16 () {56 vars in 1135 bytes} [Mon May 23 12:19:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:57.468932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350ad55-ba49-4491-b8cd-38709546235a None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:57.503593 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:57.514861 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b423b88-abb5-45a0-bc61-024b06e68936 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:57.554949 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ad7c58-c828-4433-ab26-5b580e08cf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:57.556058 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad7c58-c828-4433-ab26-5b580e08cf95 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:57.556550 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad7c58-c828-4433-ab26-5b580e08cf95 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:57.556806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad7c58-c828-4433-ab26-5b580e08cf95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:57.563947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ad7c58-c828-4433-ab26-5b580e08cf95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:57.565891 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ad7c58-c828-4433-ab26-5b580e08cf95 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:57.580312 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1350ad55-ba49-4491-b8cd-38709546235a None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:57.580586 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-37ad7c58-c828-4433-ab26-5b580e08cf95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:57.580586 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:57.580586 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:57.580586 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:57.580586 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:57.580586 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:57.580586 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:57.580586 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:57.580586 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:57.580586 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:57.580586 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:57.580586 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:57.580586 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:19:57.580586 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:19:57.580586 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:19:57.581517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:19:57.581517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:57.581517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:57.581517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:57.581517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:57.581517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:57.581517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:57.581517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:57.581517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:57.581517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:57.581517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:57.581517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:57.581517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:57.581517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:57.581517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:57.581517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:57.581517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:57.582439 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:19:57.582439 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:19:57.582439 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:19:57.582439 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:19:57.582439 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:19:57.582439 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:19:57.582439 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:57.582439 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:57.582439 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 425/856] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:57.583519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ad7c58-c828-4433-ab26-5b580e08cf95 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:57.623990 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02803b14-e5cc-41e9-aae1-4cc84f1d4109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:57.624730 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02803b14-e5cc-41e9-aae1-4cc84f1d4109 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:57.625009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02803b14-e5cc-41e9-aae1-4cc84f1d4109 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:57.625337 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02803b14-e5cc-41e9-aae1-4cc84f1d4109 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:57.626039 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02803b14-e5cc-41e9-aae1-4cc84f1d4109 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:57.626342 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02803b14-e5cc-41e9-aae1-4cc84f1d4109 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:57.627717 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02803b14-e5cc-41e9-aae1-4cc84f1d4109 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:57.633159 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 432/857] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 12:19:57 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:57.645883 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02803b14-e5cc-41e9-aae1-4cc84f1d4109 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:57.676598 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:57.704299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ad7c58-c828-4433-ab26-5b580e08cf95 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:57.745348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec85ec6c-43f2-4dd4-9efd-c71edca88cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:57.746138 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85ec6c-43f2-4dd4-9efd-c71edca88cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:57.746599 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85ec6c-43f2-4dd4-9efd-c71edca88cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:57.746818 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85ec6c-43f2-4dd4-9efd-c71edca88cb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:57.769724 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:57.818701 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02803b14-e5cc-41e9-aae1-4cc84f1d4109 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:57.834842 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec85ec6c-43f2-4dd4-9efd-c71edca88cb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:57.836741 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec85ec6c-43f2-4dd4-9efd-c71edca88cb6 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:57.858732 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a9c2a-9489-49a0-806f-45587e1a4fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:57.860077 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a9c2a-9489-49a0-806f-45587e1a4fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:57.860539 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a9c2a-9489-49a0-806f-45587e1a4fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:57.860959 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a9c2a-9489-49a0-806f-45587e1a4fb6 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:57.861883 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a9c2a-9489-49a0-806f-45587e1a4fb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:57.862333 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a9c2a-9489-49a0-806f-45587e1a4fb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:57.864466 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a9c2a-9489-49a0-806f-45587e1a4fb6 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:57.878744 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 433/858] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:19:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:57.889200 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a9c2a-9489-49a0-806f-45587e1a4fb6 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:57.964265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 426/859] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:19:57 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 12:19:57.969643 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec85ec6c-43f2-4dd4-9efd-c71edca88cb6 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:57.970486 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:58.021354 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a9c2a-9489-49a0-806f-45587e1a4fb6 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:58.045899 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:58.074110 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac2474-16dd-4a7f-82a5-11c49a8f123d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:58.075643 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac2474-16dd-4a7f-82a5-11c49a8f123d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:58.076139 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac2474-16dd-4a7f-82a5-11c49a8f123d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:58.077101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac2474-16dd-4a7f-82a5-11c49a8f123d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:58.078525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac2474-16dd-4a7f-82a5-11c49a8f123d None admin] RBAC: Adding query filter params (role.id=81d82471e2be4f0aa8164b75c16a9718, scope.project.id=839585501f414e1ab02501422510e54f, user.id=733f8296ec3d4795a1140c1e1cb6eb95) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:58.079031 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac2474-16dd-4a7f-82a5-11c49a8f123d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:58.081194 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac2474-16dd-4a7f-82a5-11c49a8f123d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:58.096212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 434/860] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:19:57 2022] GET /identity/v3/role_assignments?user.id=733f8296ec3d4795a1140c1e1cb6eb95&scope.project.id=839585501f414e1ab02501422510e54f&role.id=81d82471e2be4f0aa8164b75c16a9718 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:58.108550 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec85ec6c-43f2-4dd4-9efd-c71edca88cb6 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:58.149433 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f9094-3ef4-4ba4-863c-45294f87fa5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:58.150240 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f9094-3ef4-4ba4-863c-45294f87fa5e None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:58.150521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f9094-3ef4-4ba4-863c-45294f87fa5e None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:58.150792 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f9094-3ef4-4ba4-863c-45294f87fa5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:58.163160 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f9094-3ef4-4ba4-863c-45294f87fa5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:58.164867 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f9094-3ef4-4ba4-863c-45294f87fa5e None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:58.186625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-bd3f9094-3ef4-4ba4-863c-45294f87fa5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:58.186625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:58.186625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:58.186625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:58.186625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:58.186625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:58.186625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:58.186625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:58.186625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:58.186625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:58.186625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:58.186625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:58.186625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:19:58.186625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:19:58.186625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:19:58.187748 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:19:58.187748 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:58.187748 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:58.187748 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:58.187748 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:58.187748 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:58.187748 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:58.187748 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:58.187748 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:58.187748 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:58.187748 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:58.187748 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:58.187748 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:58.187748 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:58.187748 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:58.187748 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:58.187748 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:58.188720 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:19:58.188720 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:19:58.188720 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:19:58.188720 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:58.188720 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 427/861] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:19:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:19:58.195299 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac2474-16dd-4a7f-82a5-11c49a8f123d None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:58.205854 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776add6-4db9-49d9-a7d2-dc6d5d916076 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:58.206174 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776add6-4db9-49d9-a7d2-dc6d5d916076 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:58.206416 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776add6-4db9-49d9-a7d2-dc6d5d916076 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:58.208199 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 428/862] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:19:58.213772 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf4ecb-64c5-48a1-805a-2e88506cfbe1 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:58.214063 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf4ecb-64c5-48a1-805a-2e88506cfbe1 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:58.214342 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf4ecb-64c5-48a1-805a-2e88506cfbe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:58.235156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-c2cf4ecb-64c5-48a1-805a-2e88506cfbe1 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:19:58.237088 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-c2cf4ecb-64c5-48a1-805a-2e88506cfbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:19:58.329157 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac2474-16dd-4a7f-82a5-11c49a8f123d None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:58.330857 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 429/863] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:19:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 12:19:58.354376 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf4ecb-64c5-48a1-805a-2e88506cfbe1 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:58.373421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4bdc3c-ce92-48b9-80c7-18a2a9a2f218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:58.374251 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4bdc3c-ce92-48b9-80c7-18a2a9a2f218 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:58.374662 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4bdc3c-ce92-48b9-80c7-18a2a9a2f218 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:58.374953 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4bdc3c-ce92-48b9-80c7-18a2a9a2f218 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:58.375845 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4bdc3c-ce92-48b9-80c7-18a2a9a2f218 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:58.376159 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4bdc3c-ce92-48b9-80c7-18a2a9a2f218 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:58.377880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4bdc3c-ce92-48b9-80c7-18a2a9a2f218 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:58.380078 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:58.391827 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:58.394280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 435/864] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:19:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:58.404213 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4bdc3c-ce92-48b9-80c7-18a2a9a2f218 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:58.498462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf4ecb-64c5-48a1-805a-2e88506cfbe1 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:58.526399 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4bdc3c-ce92-48b9-80c7-18a2a9a2f218 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:58.548038 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20423acc-8f9f-4342-a77d-43caf03bd8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:58.548893 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20423acc-8f9f-4342-a77d-43caf03bd8d9 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:58.549181 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20423acc-8f9f-4342-a77d-43caf03bd8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:58.549993 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20423acc-8f9f-4342-a77d-43caf03bd8d9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:58.550671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20423acc-8f9f-4342-a77d-43caf03bd8d9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:58.552673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20423acc-8f9f-4342-a77d-43caf03bd8d9 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:58.564429 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43efc4ee-5fb2-45c3-b32d-6b70a2f7538f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:58.565321 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efc4ee-5fb2-45c3-b32d-6b70a2f7538f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:58.565612 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efc4ee-5fb2-45c3-b32d-6b70a2f7538f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:58.565894 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efc4ee-5fb2-45c3-b32d-6b70a2f7538f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:58.568623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-20423acc-8f9f-4342-a77d-43caf03bd8d9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 12:19:58.568623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:58.568623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 12:19:58.568623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 12:19:58.568623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 12:19:58.568623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 12:19:58.568623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 12:19:58.568623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:19:58.568623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 12:19:58.568623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:58.568623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:19:58.568623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:58.568623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:58.568623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:58.568623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:58.568623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:58.569533 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:58.569533 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:58.569533 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:58.569533 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:58.569533 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:58.569533 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:58.569533 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:58.569533 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 12:19:58.569533 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 12:19:58.569533 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 12:19:58.569533 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 12:19:58.569533 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:58.569533 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:58.569533 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:58.569533 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:58.569533 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:58.570395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:58.570395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:58.570395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:19:58.570395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:58.570395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:58.570395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:58.570395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:58.570395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:58.570395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:58.570395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:58.570395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:58.570395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 12:19:58.570395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:19:58.570395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 12:19:58.570395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:19:58.571958 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 430/865] 10.222.0.16 () {56 vars in 1132 bytes} [Mon May 23 12:19:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:19:58.575246 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43efc4ee-5fb2-45c3-b32d-6b70a2f7538f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:58.576488 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20423acc-8f9f-4342-a77d-43caf03bd8d9 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:58.577382 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43efc4ee-5fb2-45c3-b32d-6b70a2f7538f None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:58.593069 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-43efc4ee-5fb2-45c3-b32d-6b70a2f7538f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:58.593069 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:19:58.593069 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:19:58.593069 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:19:58.593069 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:19:58.593069 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:19:58.593069 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:19:58.593069 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:19:58.593069 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:19:58.593069 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:19:58.593069 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:19:58.593069 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:19:58.593069 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:19:58.593069 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:19:58.593069 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:19:58.595109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:19:58.595109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:19:58.595109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:19:58.595109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:19:58.595109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:19:58.595109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:19:58.595109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:19:58.595109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:19:58.595109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:19:58.595109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:19:58.595109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:19:58.595109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:19:58.595109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:19:58.595109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:19:58.595109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:19:58.595109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:19:58.595109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:19:58.596731 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:19:58.596731 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:19:58.596731 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:19:58.596731 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:19:58.596731 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:19:58.596731 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:19:58.596731 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:19:58.596731 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:19:58.596731 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 436/866] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:19:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:19:58.601384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43efc4ee-5fb2-45c3-b32d-6b70a2f7538f None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:58.671108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:58.685151 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:58.691518 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20423acc-8f9f-4342-a77d-43caf03bd8d9 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:58.722592 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43efc4ee-5fb2-45c3-b32d-6b70a2f7538f None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:58.729411 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e0a00-ab27-4683-a8d1-c45f8a63938d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:58.730291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e0a00-ab27-4683-a8d1-c45f8a63938d None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:58.730728 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e0a00-ab27-4683-a8d1-c45f8a63938d None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:58.731153 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e0a00-ab27-4683-a8d1-c45f8a63938d None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:58.732011 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e0a00-ab27-4683-a8d1-c45f8a63938d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:58.732629 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e0a00-ab27-4683-a8d1-c45f8a63938d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:58.734283 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e0a00-ab27-4683-a8d1-c45f8a63938d None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:58.745109 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 431/867] 10.222.0.16 () {56 vars in 1126 bytes} [Mon May 23 12:19:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:19:58.746929 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e0a00-ab27-4683-a8d1-c45f8a63938d None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:58.759727 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa807e-d923-46ad-9209-5b733e0d05a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:58.760608 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa807e-d923-46ad-9209-5b733e0d05a7 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:58.760766 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa807e-d923-46ad-9209-5b733e0d05a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:58.760998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa807e-d923-46ad-9209-5b733e0d05a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:58.761788 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baa807e-d923-46ad-9209-5b733e0d05a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:58.762103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baa807e-d923-46ad-9209-5b733e0d05a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:58.763540 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baa807e-d923-46ad-9209-5b733e0d05a7 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:58.777323 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 437/868] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:19:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:58.783313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa807e-d923-46ad-9209-5b733e0d05a7 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:19:58.884436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e0a00-ab27-4683-a8d1-c45f8a63938d None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:58.924507 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa807e-d923-46ad-9209-5b733e0d05a7 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:19:58.932452 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdaccf8-c2d1-4044-a035-87ef0d003781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:58.933207 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdaccf8-c2d1-4044-a035-87ef0d003781 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:58.933582 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdaccf8-c2d1-4044-a035-87ef0d003781 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:58.933936 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdaccf8-c2d1-4044-a035-87ef0d003781 None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:58.934727 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdaccf8-c2d1-4044-a035-87ef0d003781 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:58.935105 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdaccf8-c2d1-4044-a035-87ef0d003781 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:58.937110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdaccf8-c2d1-4044-a035-87ef0d003781 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:58.948341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 432/869] 10.222.0.16 () {56 vars in 1126 bytes} [Mon May 23 12:19:58 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:19:58.954698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:19:58.975665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45090445-83a2-4f13-a370-45495fedcc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:19:58.976696 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45090445-83a2-4f13-a370-45495fedcc10 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:19:58.977159 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45090445-83a2-4f13-a370-45495fedcc10 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:19:58.977640 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45090445-83a2-4f13-a370-45495fedcc10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:19:58.978674 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45090445-83a2-4f13-a370-45495fedcc10 None admin] RBAC: Adding query filter params (role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8, scope.project.id=839585501f414e1ab02501422510e54f, user.id=71fb9a79ac494635804b93d4a0048205) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:19:58.979176 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45090445-83a2-4f13-a370-45495fedcc10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:19:58.980855 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45090445-83a2-4f13-a370-45495fedcc10 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:19:58.988300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 438/870] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:19:58 2022] GET /identity/v3/role_assignments?user.id=71fb9a79ac494635804b93d4a0048205&scope.project.id=839585501f414e1ab02501422510e54f&role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:00.014345 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e449ab-537d-46c6-b8cd-ceac356550b7 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:00.014865 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e449ab-537d-46c6-b8cd-ceac356550b7 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:00.014942 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e449ab-537d-46c6-b8cd-ceac356550b7 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:00.016446 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 433/871] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:00.032692 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639965f-12c7-40be-ace8-c9838ff4faac None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:00.033529 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639965f-12c7-40be-ace8-c9838ff4faac None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:00.034007 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639965f-12c7-40be-ace8-c9838ff4faac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:00.052236 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a639965f-12c7-40be-ace8-c9838ff4faac None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:00.053916 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a639965f-12c7-40be-ace8-c9838ff4faac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:00.132270 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 439/872] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:00 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:20:00.151286 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e449ab-537d-46c6-b8cd-ceac356550b7 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:00.181438 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:00.265157 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e449ab-537d-46c6-b8cd-ceac356550b7 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:00.300487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a93a7d-2f80-4034-9f29-25d3c80ce7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:00.301158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a93a7d-2f80-4034-9f29-25d3c80ce7e7 None admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:00.301476 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a93a7d-2f80-4034-9f29-25d3c80ce7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:00.301767 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a93a7d-2f80-4034-9f29-25d3c80ce7e7 None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:00.302506 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a93a7d-2f80-4034-9f29-25d3c80ce7e7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:00.303905 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a93a7d-2f80-4034-9f29-25d3c80ce7e7 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:00.321695 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 434/873] 10.222.0.16 () {60 vars in 1141 bytes} [Mon May 23 12:20:00 2022] POST /identity/v3/services => generated 251 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:20:00.520927 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec7cb9-e601-466c-bbb3-26c59fbb8af3 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:00.521364 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec7cb9-e601-466c-bbb3-26c59fbb8af3 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:00.521678 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec7cb9-e601-466c-bbb3-26c59fbb8af3 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:00.523317 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 440/874] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:00.528567 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863c070-3f9b-491b-9554-0f9907d43195 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:00.528863 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863c070-3f9b-491b-9554-0f9907d43195 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:00.529147 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863c070-3f9b-491b-9554-0f9907d43195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:00.547976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-7863c070-3f9b-491b-9554-0f9907d43195 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:00.549221 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-7863c070-3f9b-491b-9554-0f9907d43195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:00.629202 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 435/875] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:00 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:20:00.652282 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec7cb9-e601-466c-bbb3-26c59fbb8af3 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:00.761723 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec7cb9-e601-466c-bbb3-26c59fbb8af3 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:00.772331 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:00.797865 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad559f-45b6-4ad9-b880-2a4ea56d6234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:00.798688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad559f-45b6-4ad9-b880-2a4ea56d6234 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:00.798980 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad559f-45b6-4ad9-b880-2a4ea56d6234 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:00.799316 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad559f-45b6-4ad9-b880-2a4ea56d6234 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:00.811123 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ad559f-45b6-4ad9-b880-2a4ea56d6234 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:20:00.820632 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-69ad559f-45b6-4ad9-b880-2a4ea56d6234 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 12:20:00.820632 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:00.820632 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:00.820632 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:00.820632 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:00.820632 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:00.820632 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:00.820632 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:00.820632 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:00.820632 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:00.820632 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:00.820632 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:00.820632 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:20:00.820632 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:20:00.820632 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:20:00.820632 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:20:00.821587 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:20:00.821587 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:20:00.821587 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:20:00.821587 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:20:00.821587 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:00.821587 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:00.821587 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:00.821587 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:00.821587 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:00.821587 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:00.821587 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:00.821587 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:20:00.821587 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:00.821587 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:00.821587 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:00.821587 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:00.822455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:00.822455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:00.822455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:00.822455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:00.822455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:20:00.822455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:20:00.822455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:20:00.822455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:20:00.822455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:20:00.822455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:20:00.822455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 12:20:00.822455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:00.823324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 441/876] 10.222.0.16 () {56 vars in 1120 bytes} [Mon May 23 12:20:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:00.829121 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863c070-3f9b-491b-9554-0f9907d43195 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:00.841489 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:00.936121 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863c070-3f9b-491b-9554-0f9907d43195 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:00.971125 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3e629-be40-4f80-b060-ce0c9eb8db1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:00.971796 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3e629-be40-4f80-b060-ce0c9eb8db1e None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:00.972080 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3e629-be40-4f80-b060-ce0c9eb8db1e None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:00.972388 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3e629-be40-4f80-b060-ce0c9eb8db1e None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:00.973141 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a3e629-be40-4f80-b060-ce0c9eb8db1e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:00.974188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a3e629-be40-4f80-b060-ce0c9eb8db1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:00.975574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a3e629-be40-4f80-b060-ce0c9eb8db1e None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:00.988370 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 436/877] 10.222.0.16 () {56 vars in 1115 bytes} [Mon May 23 12:20:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:00.993242 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad559f-45b6-4ad9-b880-2a4ea56d6234 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:01.102702 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad559f-45b6-4ad9-b880-2a4ea56d6234 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:01.140888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ebd1f-c798-4fc8-9075-7a1937887f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:01.141993 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ebd1f-c798-4fc8-9075-7a1937887f39 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:01.142460 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ebd1f-c798-4fc8-9075-7a1937887f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:01.142917 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ebd1f-c798-4fc8-9075-7a1937887f39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:01.155086 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ebd1f-c798-4fc8-9075-7a1937887f39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:01.156836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ebd1f-c798-4fc8-9075-7a1937887f39 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:01.174655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-cc3ebd1f-c798-4fc8-9075-7a1937887f39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:20:01.174655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:01.174655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:01.174655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:01.174655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:01.174655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:01.174655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:01.174655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:01.174655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:01.174655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:01.174655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:01.174655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:01.174655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:20:01.174655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:20:01.174655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:20:01.175590 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:20:01.175590 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:01.175590 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:01.175590 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:01.175590 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:01.175590 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:01.175590 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:01.175590 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:01.175590 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:20:01.175590 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:01.175590 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:01.175590 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:01.175590 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:01.175590 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:01.175590 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:01.175590 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:01.175590 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:01.176784 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:20:01.176784 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:20:01.176784 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:20:01.176784 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:01.178875 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 442/878] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:20:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:01.184895 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3e629-be40-4f80-b060-ce0c9eb8db1e None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:01.264787 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:01.298072 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3e629-be40-4f80-b060-ce0c9eb8db1e None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:01.299069 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a285d7-9c5c-43ba-a05f-b65daed4c854 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:01.299360 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a285d7-9c5c-43ba-a05f-b65daed4c854 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:01.299640 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a285d7-9c5c-43ba-a05f-b65daed4c854 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:01.301325 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 443/879] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:20:01.306547 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402094e-4abe-46b6-b9f9-802b8aac9734 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:01.306845 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402094e-4abe-46b6-b9f9-802b8aac9734 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:01.307130 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c402094e-4abe-46b6-b9f9-802b8aac9734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:01.325538 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-c402094e-4abe-46b6-b9f9-802b8aac9734 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:01.326707 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c402094e-4abe-46b6-b9f9-802b8aac9734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:01.336553 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d64495-8182-425b-8545-b014bce6c9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:01.337552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d64495-8182-425b-8545-b014bce6c9fe None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:01.338018 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d64495-8182-425b-8545-b014bce6c9fe None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:01.338503 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d64495-8182-425b-8545-b014bce6c9fe None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:01.339554 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d64495-8182-425b-8545-b014bce6c9fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:01.340043 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d64495-8182-425b-8545-b014bce6c9fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:01.341771 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d64495-8182-425b-8545-b014bce6c9fe None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:01.342383 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:01.357304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 437/880] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:20:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:01.362892 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d64495-8182-425b-8545-b014bce6c9fe None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:01.400295 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 444/881] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:01 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:20:01.411401 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a774867-b8ca-4298-bf67-e2f5743a0f37 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:01.411704 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a774867-b8ca-4298-bf67-e2f5743a0f37 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:01.411986 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a774867-b8ca-4298-bf67-e2f5743a0f37 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:01.413653 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 445/882] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:20:01.420801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c4fe3-3bf7-426f-8ad5-e59b071db40e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:01.421266 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c4fe3-3bf7-426f-8ad5-e59b071db40e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:01.421552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c4fe3-3bf7-426f-8ad5-e59b071db40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:01.444877 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a71c4fe3-3bf7-426f-8ad5-e59b071db40e None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:01.446074 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a71c4fe3-3bf7-426f-8ad5-e59b071db40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:01.476370 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d64495-8182-425b-8545-b014bce6c9fe None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:01.507415 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:01.512814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d0163-5fbb-4d23-a895-f9c4797d1df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:01.513603 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d0163-5fbb-4d23-a895-f9c4797d1df7 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:01.513888 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d0163-5fbb-4d23-a895-f9c4797d1df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:01.514155 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d0163-5fbb-4d23-a895-f9c4797d1df7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:01.521597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d0163-5fbb-4d23-a895-f9c4797d1df7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:01.522965 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 446/883] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:01 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:20:01.523430 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d0163-5fbb-4d23-a895-f9c4797d1df7 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:01.523781 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c4fe3-3bf7-426f-8ad5-e59b071db40e None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:01.550707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-7c4d0163-5fbb-4d23-a895-f9c4797d1df7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 12:20:01.550707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:01.550707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:01.550707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:01.550707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:01.550707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:01.550707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:01.550707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:01.550707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:01.550707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:01.550707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:01.550707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:01.550707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:20:01.550707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:20:01.550707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:20:01.550707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:20:01.552194 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:01.552194 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:01.552194 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:20:01.552194 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:20:01.552194 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:20:01.552194 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:20:01.552194 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:01.552194 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:01.552194 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:01.552194 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:01.552194 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:01.552194 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:20:01.552194 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:01.552194 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:01.552194 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:01.552194 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:01.552194 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:01.552194 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:01.553609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:01.553609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:01.553609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:20:01.553609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:20:01.553609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:20:01.553609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:20:01.553609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:20:01.553609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:20:01.553609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 12:20:01.553609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:01.553609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 438/884] 10.222.0.16 () {56 vars in 1117 bytes} [Mon May 23 12:20:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:20:01.553609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d0163-5fbb-4d23-a895-f9c4797d1df7 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:01.637084 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:01.657337 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c4fe3-3bf7-426f-8ad5-e59b071db40e None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:01.673296 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d0163-5fbb-4d23-a895-f9c4797d1df7 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:01.696603 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c303621-a755-45a0-a799-d1173ecf415e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:01.697764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c303621-a755-45a0-a799-d1173ecf415e None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:01.698623 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c303621-a755-45a0-a799-d1173ecf415e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:01.699340 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c303621-a755-45a0-a799-d1173ecf415e None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:01.700589 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c303621-a755-45a0-a799-d1173ecf415e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:01.702837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c303621-a755-45a0-a799-d1173ecf415e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:01.711560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e7446-37bf-426d-96f8-de47ff57e39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:01.713141 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e7446-37bf-426d-96f8-de47ff57e39c None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:01.713392 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e7446-37bf-426d-96f8-de47ff57e39c None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:01.713536 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e7446-37bf-426d-96f8-de47ff57e39c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:01.714543 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e7446-37bf-426d-96f8-de47ff57e39c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:01.716440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e7446-37bf-426d-96f8-de47ff57e39c None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:01.724353 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 447/885] 10.222.0.16 () {60 vars in 1141 bytes} [Mon May 23 12:20:01 2022] POST /identity/v3/services => generated 269 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:20:01.725457 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c303621-a755-45a0-a799-d1173ecf415e None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:01.729898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-a79e7446-37bf-426d-96f8-de47ff57e39c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 12:20:01.729898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:01.729898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 12:20:01.729898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 12:20:01.729898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 12:20:01.729898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 12:20:01.729898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 12:20:01.729898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:20:01.729898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 12:20:01.729898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:01.729898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:20:01.729898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:01.729898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:01.729898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:01.729898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:01.729898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:01.729898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:01.731127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:01.731127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:01.731127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:01.731127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:01.731127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:01.731127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:01.731127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 12:20:01.731127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 12:20:01.731127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 12:20:01.731127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 12:20:01.731127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:01.731127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:01.731127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:01.731127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:01.731127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:01.731127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:01.732049 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:01.732049 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:20:01.732049 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:01.732049 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:01.732049 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:01.732049 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:01.732049 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:01.732049 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:01.732049 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:01.732049 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:01.732049 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 12:20:01.732049 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:20:01.732049 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 12:20:01.732049 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:01.732049 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 439/886] 10.222.0.16 () {56 vars in 1126 bytes} [Mon May 23 12:20:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:01.738686 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e7446-37bf-426d-96f8-de47ff57e39c None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:01.830004 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:01.885269 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e7446-37bf-426d-96f8-de47ff57e39c None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:01.889426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c303621-a755-45a0-a799-d1173ecf415e None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:01.925209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d0c5a-963f-4103-a1cb-7145fe9a5518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:01.926307 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d0c5a-963f-4103-a1cb-7145fe9a5518 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:01.927331 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d0c5a-963f-4103-a1cb-7145fe9a5518 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:01.927908 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d0c5a-963f-4103-a1cb-7145fe9a5518 None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:01.928959 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d0c5a-963f-4103-a1cb-7145fe9a5518 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:01.929483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d0c5a-963f-4103-a1cb-7145fe9a5518 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:01.931195 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d0c5a-963f-4103-a1cb-7145fe9a5518 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:01.935743 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86445282-cb96-4c4e-bc30-4d425a8ed3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:01.936457 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86445282-cb96-4c4e-bc30-4d425a8ed3ec None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:01.936740 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86445282-cb96-4c4e-bc30-4d425a8ed3ec None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:01.936848 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 440/887] 10.222.0.16 () {56 vars in 1122 bytes} [Mon May 23 12:20:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:01.940582 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86445282-cb96-4c4e-bc30-4d425a8ed3ec None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:01.941719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86445282-cb96-4c4e-bc30-4d425a8ed3ec None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:01.942650 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d0c5a-963f-4103-a1cb-7145fe9a5518 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:01.942903 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86445282-cb96-4c4e-bc30-4d425a8ed3ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:01.944568 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86445282-cb96-4c4e-bc30-4d425a8ed3ec None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:01.947969 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:01.966201 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 448/888] 10.222.0.16 () {56 vars in 1149 bytes} [Mon May 23 12:20:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:01.971549 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86445282-cb96-4c4e-bc30-4d425a8ed3ec None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:02.073774 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d0c5a-963f-4103-a1cb-7145fe9a5518 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:02.096421 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:02.109529 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86445282-cb96-4c4e-bc30-4d425a8ed3ec None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:02.117040 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7e0cd-cb51-4107-af0a-a174bf3ef976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:02.117765 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7e0cd-cb51-4107-af0a-a174bf3ef976 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:02.118052 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7e0cd-cb51-4107-af0a-a174bf3ef976 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:02.118383 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7e0cd-cb51-4107-af0a-a174bf3ef976 None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:02.119104 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7e0cd-cb51-4107-af0a-a174bf3ef976 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:02.119414 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7e0cd-cb51-4107-af0a-a174bf3ef976 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:02.120899 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7e0cd-cb51-4107-af0a-a174bf3ef976 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:02.128215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 441/889] 10.222.0.16 () {56 vars in 1122 bytes} [Mon May 23 12:20:01 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:02.159337 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9afa535-fa34-408f-98a3-79915e6f0a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:02.160422 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afa535-fa34-408f-98a3-79915e6f0a22 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:02.161803 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afa535-fa34-408f-98a3-79915e6f0a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:02.162281 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afa535-fa34-408f-98a3-79915e6f0a22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:02.235905 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:02.244941 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9afa535-fa34-408f-98a3-79915e6f0a22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:02.246860 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9afa535-fa34-408f-98a3-79915e6f0a22 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:02.360115 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 449/890] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:20:01 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 12:20:02.365828 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7e0cd-cb51-4107-af0a-a174bf3ef976 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:02.469794 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7e0cd-cb51-4107-af0a-a174bf3ef976 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:02.507742 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6777e15c-af3f-4aea-8961-b08ab9b114bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:02.508609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777e15c-af3f-4aea-8961-b08ab9b114bb None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:02.508891 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777e15c-af3f-4aea-8961-b08ab9b114bb None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:02.509168 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6777e15c-af3f-4aea-8961-b08ab9b114bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:02.522464 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6777e15c-af3f-4aea-8961-b08ab9b114bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:02.523969 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6777e15c-af3f-4aea-8961-b08ab9b114bb None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:02.542827 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-6777e15c-af3f-4aea-8961-b08ab9b114bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:20:02.542827 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:02.542827 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:02.542827 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:02.542827 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:02.542827 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:02.542827 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:02.542827 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:02.542827 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:02.542827 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:02.542827 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:02.542827 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:02.542827 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:20:02.542827 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:20:02.542827 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:20:02.543819 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:20:02.543819 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:02.543819 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:02.543819 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:02.543819 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:02.543819 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:02.543819 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:02.543819 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:02.543819 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:20:02.543819 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:02.543819 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:02.543819 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:02.543819 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:02.543819 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:02.543819 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:02.543819 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:02.543819 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:02.544777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:20:02.544777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:20:02.544777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:20:02.544777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:02.544777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 442/891] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:20:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:02.553295 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9afa535-fa34-408f-98a3-79915e6f0a22 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:02.655819 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:02.663068 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9afa535-fa34-408f-98a3-79915e6f0a22 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:02.696963 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45518a-8ae3-431d-8029-634b1e076b99 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:02.697300 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45518a-8ae3-431d-8029-634b1e076b99 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:02.697535 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45518a-8ae3-431d-8029-634b1e076b99 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:02.699116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 443/892] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:02.701814 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e29284-a2e3-4b5c-9af4-6aa66e30132a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:02.702721 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e29284-a2e3-4b5c-9af4-6aa66e30132a None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:02.703223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e29284-a2e3-4b5c-9af4-6aa66e30132a None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:02.703732 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e29284-a2e3-4b5c-9af4-6aa66e30132a None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:02.704020 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887ce55-d3e3-4362-bd4f-d94f723dfaa8 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:02.704358 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887ce55-d3e3-4362-bd4f-d94f723dfaa8 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:02.704644 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887ce55-d3e3-4362-bd4f-d94f723dfaa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:02.705347 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e29284-a2e3-4b5c-9af4-6aa66e30132a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:02.705869 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e29284-a2e3-4b5c-9af4-6aa66e30132a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:02.708207 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e29284-a2e3-4b5c-9af4-6aa66e30132a None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:02.721409 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 450/893] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:02.726341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-1887ce55-d3e3-4362-bd4f-d94f723dfaa8 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:02.726341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-1887ce55-d3e3-4362-bd4f-d94f723dfaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:02.727608 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e29284-a2e3-4b5c-9af4-6aa66e30132a None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:02.763402 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:02.822275 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 444/894] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:02 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 12:20:02.844393 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887ce55-d3e3-4362-bd4f-d94f723dfaa8 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:02.846315 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e29284-a2e3-4b5c-9af4-6aa66e30132a None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:02.890995 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5576a72-dbaa-4ea5-b404-f1a115291364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:02.892013 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5576a72-dbaa-4ea5-b404-f1a115291364 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:02.892430 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5576a72-dbaa-4ea5-b404-f1a115291364 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:02.892806 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5576a72-dbaa-4ea5-b404-f1a115291364 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:02.902536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5576a72-dbaa-4ea5-b404-f1a115291364 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:02.904910 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5576a72-dbaa-4ea5-b404-f1a115291364 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:02.925498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-a5576a72-dbaa-4ea5-b404-f1a115291364 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:20:02.925498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:02.925498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:02.925498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:02.925498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:02.925498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:02.925498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:02.925498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:02.925498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:02.925498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:02.925498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:02.925498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:02.925498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:20:02.925498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:20:02.925498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:20:02.927079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:20:02.927079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:02.927079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:02.927079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:02.927079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:02.927079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:02.927079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:02.927079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:02.927079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:20:02.927079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:02.927079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:02.927079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:02.927079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:02.927079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:02.927079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:02.927079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:02.927079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:02.927982 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:20:02.927982 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:20:02.927982 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:20:02.927982 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:20:02.927982 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:20:02.927982 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:20:02.927982 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:20:02.927982 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:02.932191 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 451/895] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:20:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:02.934626 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5576a72-dbaa-4ea5-b404-f1a115291364 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:02.955031 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:02.977203 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887ce55-d3e3-4362-bd4f-d94f723dfaa8 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:03.023583 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ffdbf4-6884-4dde-a94d-ba217913d5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:03.024710 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ffdbf4-6884-4dde-a94d-ba217913d5b7 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:03.024994 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ffdbf4-6884-4dde-a94d-ba217913d5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:03.025271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ffdbf4-6884-4dde-a94d-ba217913d5b7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:03.025937 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ffdbf4-6884-4dde-a94d-ba217913d5b7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:03.032273 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ffdbf4-6884-4dde-a94d-ba217913d5b7 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:03.053013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-25ffdbf4-6884-4dde-a94d-ba217913d5b7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 12:20:03.053013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:03.053013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 12:20:03.053013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 12:20:03.053013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 12:20:03.053013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 12:20:03.053013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 12:20:03.053013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:20:03.053013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 12:20:03.053013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:03.053013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:20:03.053013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:03.053013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:03.053013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:03.053013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:03.053013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:03.053013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:03.054258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:03.054258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:03.054258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:03.054258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:03.054258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:03.054258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:03.054258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 12:20:03.054258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 12:20:03.054258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 12:20:03.054258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 12:20:03.054258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:03.054258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:03.054258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:03.054258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:03.054258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:03.054258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:03.055382 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:03.055382 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:20:03.055382 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:03.055382 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:03.055382 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:03.055382 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:03.055382 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:03.055382 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:03.055382 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:03.055382 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:03.055382 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 12:20:03.055382 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:20:03.055382 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 12:20:03.055382 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:03.060217 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 445/896] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 12:20:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:20:03.063542 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ffdbf4-6884-4dde-a94d-ba217913d5b7 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:03.074268 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5576a72-dbaa-4ea5-b404-f1a115291364 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:03.108289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:03.111897 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d58ca3-8ffa-44e7-8bb9-78f060d02e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:03.112677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d58ca3-8ffa-44e7-8bb9-78f060d02e0d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:03.112969 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d58ca3-8ffa-44e7-8bb9-78f060d02e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:03.113250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d58ca3-8ffa-44e7-8bb9-78f060d02e0d None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:03.114613 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d58ca3-8ffa-44e7-8bb9-78f060d02e0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:03.114925 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d58ca3-8ffa-44e7-8bb9-78f060d02e0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:03.117481 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d58ca3-8ffa-44e7-8bb9-78f060d02e0d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:03.137446 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 452/897] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:20:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:03.138557 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d58ca3-8ffa-44e7-8bb9-78f060d02e0d None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:03.218889 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ffdbf4-6884-4dde-a94d-ba217913d5b7 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:03.231876 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:03.259206 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b1f28-6dd5-4b85-ab23-976967cd3750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:03.260083 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b1f28-6dd5-4b85-ab23-976967cd3750 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:03.260497 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b1f28-6dd5-4b85-ab23-976967cd3750 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:03.260875 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39b1f28-6dd5-4b85-ab23-976967cd3750 None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:03.261691 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b1f28-6dd5-4b85-ab23-976967cd3750 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:03.262102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b1f28-6dd5-4b85-ab23-976967cd3750 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:03.263580 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39b1f28-6dd5-4b85-ab23-976967cd3750 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:03.265991 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d58ca3-8ffa-44e7-8bb9-78f060d02e0d None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:03.273857 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 446/898] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 12:20:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:03.275221 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b1f28-6dd5-4b85-ab23-976967cd3750 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:03.314621 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46407d40-909e-4658-a1f8-49c1f0bee272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:03.315529 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46407d40-909e-4658-a1f8-49c1f0bee272 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:03.315973 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46407d40-909e-4658-a1f8-49c1f0bee272 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:03.316445 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46407d40-909e-4658-a1f8-49c1f0bee272 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:03.317965 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46407d40-909e-4658-a1f8-49c1f0bee272 None admin] RBAC: Adding query filter params (role.id=3d40411f939f4ff6af2e78c756f6cc9e, scope.project.id=839585501f414e1ab02501422510e54f, user.id=733f8296ec3d4795a1140c1e1cb6eb95) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:03.318876 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46407d40-909e-4658-a1f8-49c1f0bee272 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:03.320844 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46407d40-909e-4658-a1f8-49c1f0bee272 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:03.336700 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 453/899] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:20:03 2022] GET /identity/v3/role_assignments?user.id=733f8296ec3d4795a1140c1e1cb6eb95&scope.project.id=839585501f414e1ab02501422510e54f&role.id=3d40411f939f4ff6af2e78c756f6cc9e => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:03.388153 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:03.396576 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39b1f28-6dd5-4b85-ab23-976967cd3750 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:03.438371 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8873d96-c270-4b80-a19f-49c8e2b6b431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:03.439236 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8873d96-c270-4b80-a19f-49c8e2b6b431 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:03.439565 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8873d96-c270-4b80-a19f-49c8e2b6b431 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:03.439915 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8873d96-c270-4b80-a19f-49c8e2b6b431 None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:03.441000 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8873d96-c270-4b80-a19f-49c8e2b6b431 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:03.441423 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8873d96-c270-4b80-a19f-49c8e2b6b431 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:03.443767 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8873d96-c270-4b80-a19f-49c8e2b6b431 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:03.450100 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 447/900] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 12:20:03 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:03.456523 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46407d40-909e-4658-a1f8-49c1f0bee272 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:03.580219 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46407d40-909e-4658-a1f8-49c1f0bee272 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:03.618635 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5a471-5e8a-4505-8b56-b0247db2ad42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:03.619365 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5a471-5e8a-4505-8b56-b0247db2ad42 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:03.619656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5a471-5e8a-4505-8b56-b0247db2ad42 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:03.619937 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5a471-5e8a-4505-8b56-b0247db2ad42 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:03.620874 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5a471-5e8a-4505-8b56-b0247db2ad42 None admin] RBAC: Adding query filter params (interface=public, service_id=01f2bd5089f74b47b470b0e7ec457e3d, region_id=RegionOne) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:03.621196 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5a471-5e8a-4505-8b56-b0247db2ad42 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:03.622583 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5a471-5e8a-4505-8b56-b0247db2ad42 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:03.628130 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 454/901] 10.222.0.16 () {56 vars in 1251 bytes} [Mon May 23 12:20:03 2022] GET /identity/v3/endpoints?service_id=01f2bd5089f74b47b470b0e7ec457e3d&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:03.637273 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8873d96-c270-4b80-a19f-49c8e2b6b431 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:03.741619 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8873d96-c270-4b80-a19f-49c8e2b6b431 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:03.778572 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30da920-79fd-4c96-92b8-9e3c4a00fa62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:03.779664 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30da920-79fd-4c96-92b8-9e3c4a00fa62 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:03.780190 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30da920-79fd-4c96-92b8-9e3c4a00fa62 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:03.780714 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30da920-79fd-4c96-92b8-9e3c4a00fa62 None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:03.781596 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30da920-79fd-4c96-92b8-9e3c4a00fa62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:03.783181 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30da920-79fd-4c96-92b8-9e3c4a00fa62 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:03.790635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 448/902] 10.222.0.16 () {56 vars in 1087 bytes} [Mon May 23 12:20:03 2022] GET /identity/v3/services => generated 1268 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:20:04.113017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3661f-9b26-4ed8-a129-5e6c85a1b17b None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:04.113561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3661f-9b26-4ed8-a129-5e6c85a1b17b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:04.113561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3661f-9b26-4ed8-a129-5e6c85a1b17b None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:04.115218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 455/903] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:04.120484 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334b53a-2bcb-412f-8ccf-5cf0a0438758 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:04.120777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334b53a-2bcb-412f-8ccf-5cf0a0438758 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:04.121054 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334b53a-2bcb-412f-8ccf-5cf0a0438758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:04.138522 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-6334b53a-2bcb-412f-8ccf-5cf0a0438758 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:04.139721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-6334b53a-2bcb-412f-8ccf-5cf0a0438758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:04.162576 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:04.212628 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 449/904] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:20:04.235470 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d3661f-9b26-4ed8-a129-5e6c85a1b17b None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:04.238586 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:04.348888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d3661f-9b26-4ed8-a129-5e6c85a1b17b None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:04.386750 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99213204-91a3-4893-85cd-99c5485b8767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:04.387573 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99213204-91a3-4893-85cd-99c5485b8767 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:04.387872 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99213204-91a3-4893-85cd-99c5485b8767 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:04.388152 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99213204-91a3-4893-85cd-99c5485b8767 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:04.388842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99213204-91a3-4893-85cd-99c5485b8767 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:04.390203 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99213204-91a3-4893-85cd-99c5485b8767 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:04.402852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-99213204-91a3-4893-85cd-99c5485b8767 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 12:20:04.402852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:04.402852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 12:20:04.402852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 12:20:04.402852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 12:20:04.402852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 12:20:04.402852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 12:20:04.402852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:20:04.402852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 12:20:04.402852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:04.402852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:20:04.402852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:04.402852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:04.402852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:04.402852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:04.402852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:04.403853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:04.403853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:04.403853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:04.403853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:04.403853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:04.403853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:04.403853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:04.403853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 12:20:04.403853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 12:20:04.403853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 12:20:04.403853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 12:20:04.403853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:04.403853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:04.403853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:04.403853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:04.403853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:04.404755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:04.404755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:04.404755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:20:04.404755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:04.404755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:04.404755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:04.404755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:04.404755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:04.404755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:04.404755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:04.404755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:04.404755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 12:20:04.404755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:20:04.404755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 12:20:04.404755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:04.404755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 456/905] 10.222.0.16 () {56 vars in 1132 bytes} [Mon May 23 12:20:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:04.410179 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334b53a-2bcb-412f-8ccf-5cf0a0438758 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:04.510197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334b53a-2bcb-412f-8ccf-5cf0a0438758 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:04.527787 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb795f61-ce27-4d8b-a98b-0fbc21f21657 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:04.528156 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb795f61-ce27-4d8b-a98b-0fbc21f21657 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:04.528480 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb795f61-ce27-4d8b-a98b-0fbc21f21657 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:04.530103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 457/906] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:20:04.535442 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7b94d-a8dd-4981-9dd9-11530668f8f6 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:04.536289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7b94d-a8dd-4981-9dd9-11530668f8f6 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:04.536683 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7b94d-a8dd-4981-9dd9-11530668f8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:04.547994 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02595b9-8790-4f05-ae2e-75cf238749d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:04.549123 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02595b9-8790-4f05-ae2e-75cf238749d3 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:04.549632 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02595b9-8790-4f05-ae2e-75cf238749d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:04.550145 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02595b9-8790-4f05-ae2e-75cf238749d3 None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:04.551077 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02595b9-8790-4f05-ae2e-75cf238749d3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:04.551642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02595b9-8790-4f05-ae2e-75cf238749d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:04.553495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02595b9-8790-4f05-ae2e-75cf238749d3 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:04.556429 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a1b7b94d-a8dd-4981-9dd9-11530668f8f6 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:04.557910 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a1b7b94d-a8dd-4981-9dd9-11530668f8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:04.560000 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 450/907] 10.222.0.16 () {56 vars in 1126 bytes} [Mon May 23 12:20:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:04.565674 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02595b9-8790-4f05-ae2e-75cf238749d3 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:04.586097 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:04.636637 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:04.646352 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 458/908] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 12:20:04.666229 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7b94d-a8dd-4981-9dd9-11530668f8f6 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:04.686731 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02595b9-8790-4f05-ae2e-75cf238749d3 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:04.726325 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67990e6-86e8-43a2-9f0f-b39f22714b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:04.727096 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67990e6-86e8-43a2-9f0f-b39f22714b87 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:04.727387 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67990e6-86e8-43a2-9f0f-b39f22714b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:04.727686 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67990e6-86e8-43a2-9f0f-b39f22714b87 None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:04.728508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67990e6-86e8-43a2-9f0f-b39f22714b87 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:04.728879 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67990e6-86e8-43a2-9f0f-b39f22714b87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:04.730430 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67990e6-86e8-43a2-9f0f-b39f22714b87 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:04.736207 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 451/909] 10.222.0.16 () {56 vars in 1126 bytes} [Mon May 23 12:20:04 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:04.745268 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67990e6-86e8-43a2-9f0f-b39f22714b87 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:04.776486 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7b94d-a8dd-4981-9dd9-11530668f8f6 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:04.818433 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37234c3-dcb1-4141-ad39-09ff7c75ebca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:04.819107 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37234c3-dcb1-4141-ad39-09ff7c75ebca None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:04.819389 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37234c3-dcb1-4141-ad39-09ff7c75ebca None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:04.819671 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37234c3-dcb1-4141-ad39-09ff7c75ebca None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:04.820738 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37234c3-dcb1-4141-ad39-09ff7c75ebca None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:04.822395 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37234c3-dcb1-4141-ad39-09ff7c75ebca None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:04.840264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 459/910] 10.222.0.16 () {60 vars in 1141 bytes} [Mon May 23 12:20:04 2022] POST /identity/v3/services => generated 241 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:20:04.849812 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:04.876409 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67990e6-86e8-43a2-9f0f-b39f22714b87 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:04.924155 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd130af-b6cd-4647-ab6d-741749446461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:04.925241 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd130af-b6cd-4647-ab6d-741749446461 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:04.925766 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd130af-b6cd-4647-ab6d-741749446461 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:04.926414 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd130af-b6cd-4647-ab6d-741749446461 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:04.928093 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd130af-b6cd-4647-ab6d-741749446461 None admin] RBAC: Adding query filter params (interface=public, service_id=d0b1d78157a545a7b46cc5cb30b6ace9, region_id=RegionOne) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:04.928655 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd130af-b6cd-4647-ab6d-741749446461 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:04.930355 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd130af-b6cd-4647-ab6d-741749446461 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:04.939638 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 452/911] 10.222.0.16 () {56 vars in 1251 bytes} [Mon May 23 12:20:04 2022] GET /identity/v3/endpoints?service_id=d0b1d78157a545a7b46cc5cb30b6ace9&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:04.946383 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37234c3-dcb1-4141-ad39-09ff7c75ebca None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:05.015451 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:05.068664 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37234c3-dcb1-4141-ad39-09ff7c75ebca None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:05.104894 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c132a-0195-4ffb-a6f3-a145e2d18b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:05.105566 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c132a-0195-4ffb-a6f3-a145e2d18b76 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:05.105848 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c132a-0195-4ffb-a6f3-a145e2d18b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:05.106127 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c132a-0195-4ffb-a6f3-a145e2d18b76 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:05.106811 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c132a-0195-4ffb-a6f3-a145e2d18b76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:05.108308 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c132a-0195-4ffb-a6f3-a145e2d18b76 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:05.115799 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 460/912] 10.222.0.16 () {56 vars in 1087 bytes} [Mon May 23 12:20:04 2022] GET /identity/v3/services => generated 1497 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:20:05.717593 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2aaf9-8a6c-4163-85c5-f8a65f3c3a4d None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:05.718138 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2aaf9-8a6c-4163-85c5-f8a65f3c3a4d None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:05.718138 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2aaf9-8a6c-4163-85c5-f8a65f3c3a4d None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:05.719771 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 453/913] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:05.725142 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c8016-211a-43f1-9499-0dabdda54bd5 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:05.725709 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c8016-211a-43f1-9499-0dabdda54bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:05.726203 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c8016-211a-43f1-9499-0dabdda54bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:05.743637 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-927c8016-211a-43f1-9499-0dabdda54bd5 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:05.745154 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-927c8016-211a-43f1-9499-0dabdda54bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:05.815119 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 461/914] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:20:05.838582 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a2aaf9-8a6c-4163-85c5-f8a65f3c3a4d None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:05.958189 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a2aaf9-8a6c-4163-85c5-f8a65f3c3a4d None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:05.995068 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51864f38-a389-4d20-813f-d387792c8e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:05.995800 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51864f38-a389-4d20-813f-d387792c8e8a None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:05.996082 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51864f38-a389-4d20-813f-d387792c8e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:05.996387 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51864f38-a389-4d20-813f-d387792c8e8a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:06.005904 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:06.010398 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51864f38-a389-4d20-813f-d387792c8e8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:20:06.021462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-51864f38-a389-4d20-813f-d387792c8e8a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 12:20:06.021462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:06.021462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:06.021462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:06.021462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:06.021462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:06.021462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:06.021462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:06.021462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:06.021462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:06.021462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:06.021462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:06.021462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:20:06.021462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:20:06.021462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:20:06.021462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:20:06.022443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:20:06.022443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:20:06.022443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:20:06.022443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:20:06.022443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:06.022443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:06.022443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:06.022443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:06.022443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:06.022443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:06.022443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:06.022443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:20:06.022443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:06.022443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:06.022443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:06.022443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:06.023304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:06.023304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:06.023304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:06.023304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:06.023304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:20:06.023304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:20:06.023304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:20:06.023304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:20:06.023304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:20:06.023304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:20:06.023304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 12:20:06.023304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:06.023304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 454/915] 10.222.0.16 () {56 vars in 1120 bytes} [Mon May 23 12:20:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:06.027325 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c8016-211a-43f1-9499-0dabdda54bd5 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:06.091649 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:06.129674 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdbb78b-cf70-489b-9908-724d1e86efef None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:06.129992 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdbb78b-cf70-489b-9908-724d1e86efef None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:06.130539 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdbb78b-cf70-489b-9908-724d1e86efef None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:06.135872 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 455/916] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:06.137702 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c6ece-0adc-4993-b7f7-df3d364f595c None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:06.138205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c6ece-0adc-4993-b7f7-df3d364f595c None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:06.138691 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c6ece-0adc-4993-b7f7-df3d364f595c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:06.143672 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c8016-211a-43f1-9499-0dabdda54bd5 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:06.157664 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-807c6ece-0adc-4993-b7f7-df3d364f595c None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:06.159152 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-807c6ece-0adc-4993-b7f7-df3d364f595c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:06.183093 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06da25-1ce7-43fb-a503-86945787ce8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:06.183882 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06da25-1ce7-43fb-a503-86945787ce8d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:06.184260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06da25-1ce7-43fb-a503-86945787ce8d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:06.184609 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06da25-1ce7-43fb-a503-86945787ce8d None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:06.185558 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06da25-1ce7-43fb-a503-86945787ce8d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:06.185942 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06da25-1ce7-43fb-a503-86945787ce8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:06.187497 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06da25-1ce7-43fb-a503-86945787ce8d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:06.196929 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 462/917] 10.222.0.16 () {56 vars in 1115 bytes} [Mon May 23 12:20:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:06.204414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06da25-1ce7-43fb-a503-86945787ce8d None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:06.261528 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 456/918] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 12:20:06.291124 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c6ece-0adc-4993-b7f7-df3d364f595c None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:06.328124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06da25-1ce7-43fb-a503-86945787ce8d None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:06.342896 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:06.366158 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b2ccb-d1c5-495f-b134-273454311770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:06.366960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b2ccb-d1c5-495f-b134-273454311770 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:06.367247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b2ccb-d1c5-495f-b134-273454311770 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:06.367528 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b2ccb-d1c5-495f-b134-273454311770 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:06.378976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b2ccb-d1c5-495f-b134-273454311770 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:06.381157 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b2ccb-d1c5-495f-b134-273454311770 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:06.400258 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-412b2ccb-d1c5-495f-b134-273454311770 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:20:06.400258 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:06.400258 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:06.400258 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:06.400258 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:06.400258 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:06.400258 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:06.400258 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:06.400258 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:06.400258 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:06.400258 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:06.400258 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:06.400258 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:20:06.400258 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:20:06.400258 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:20:06.401223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:20:06.401223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:06.401223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:06.401223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:06.401223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:06.401223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:06.401223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:06.401223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:06.401223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:20:06.401223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:06.401223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:06.401223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:06.401223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:06.401223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:06.401223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:06.401223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:06.401223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:06.402139 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:20:06.402139 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:20:06.402139 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:20:06.402139 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:06.402139 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 463/919] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:20:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:06.407836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b2ccb-d1c5-495f-b134-273454311770 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:06.420083 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c6ece-0adc-4993-b7f7-df3d364f595c None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:06.456144 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:06.465738 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a9480-0adb-479c-974f-dda02eb4ced9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:06.466695 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a9480-0adb-479c-974f-dda02eb4ced9 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:06.466983 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a9480-0adb-479c-974f-dda02eb4ced9 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:06.467266 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a9480-0adb-479c-974f-dda02eb4ced9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:06.468335 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3a9480-0adb-479c-974f-dda02eb4ced9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:06.470013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3a9480-0adb-479c-974f-dda02eb4ced9 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:06.485242 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-5f3a9480-0adb-479c-974f-dda02eb4ced9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 12:20:06.485242 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:06.485242 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 12:20:06.485242 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 12:20:06.485242 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 12:20:06.485242 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 12:20:06.485242 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 12:20:06.485242 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:20:06.485242 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 12:20:06.485242 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:06.485242 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:20:06.485242 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:06.485242 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:06.485242 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:06.485242 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:06.485242 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:06.485242 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:06.486301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:06.486301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:06.486301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:06.486301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:06.486301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:06.486301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:06.486301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 12:20:06.486301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 12:20:06.486301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 12:20:06.486301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 12:20:06.486301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:06.486301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:06.486301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:06.486301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:06.486301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:06.486301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:06.487151 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:06.487151 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:20:06.487151 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:06.487151 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:06.487151 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:06.487151 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:06.487151 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:06.487151 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:06.487151 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:06.487151 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:06.487151 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 12:20:06.487151 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:20:06.487151 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 12:20:06.487151 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:06.491591 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 457/920] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 12:20:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:20:06.492647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a9480-0adb-479c-974f-dda02eb4ced9 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:06.540099 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b2ccb-d1c5-495f-b134-273454311770 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:06.588429 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f84d33-3b4d-4d58-8966-490da2500e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:06.589973 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f84d33-3b4d-4d58-8966-490da2500e3f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:06.590481 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f84d33-3b4d-4d58-8966-490da2500e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:06.591356 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f84d33-3b4d-4d58-8966-490da2500e3f None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:06.592606 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f84d33-3b4d-4d58-8966-490da2500e3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:06.593144 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f84d33-3b4d-4d58-8966-490da2500e3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:06.595476 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f84d33-3b4d-4d58-8966-490da2500e3f None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:06.618904 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a9480-0adb-479c-974f-dda02eb4ced9 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:06.619621 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 464/921] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:06.621634 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f84d33-3b4d-4d58-8966-490da2500e3f None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:06.655351 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0490e-6571-4454-b9a3-6643cbac11ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:06.655979 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:06.656217 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0490e-6571-4454-b9a3-6643cbac11ca None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:06.656603 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0490e-6571-4454-b9a3-6643cbac11ca None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:06.657109 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0490e-6571-4454-b9a3-6643cbac11ca None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:06.657828 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f0490e-6571-4454-b9a3-6643cbac11ca None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:06.658229 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f0490e-6571-4454-b9a3-6643cbac11ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:06.659743 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f0490e-6571-4454-b9a3-6643cbac11ca None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:06.671180 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 458/922] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 12:20:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:06.672365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0490e-6571-4454-b9a3-6643cbac11ca None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:06.737910 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:06.759561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f84d33-3b4d-4d58-8966-490da2500e3f None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:06.787821 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0490e-6571-4454-b9a3-6643cbac11ca None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:06.805365 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308c1f6-ce76-4354-9bde-07b2409dd826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:06.806631 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308c1f6-ce76-4354-9bde-07b2409dd826 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:06.807386 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308c1f6-ce76-4354-9bde-07b2409dd826 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:06.807853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308c1f6-ce76-4354-9bde-07b2409dd826 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:06.818235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e308c1f6-ce76-4354-9bde-07b2409dd826 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:06.820428 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e308c1f6-ce76-4354-9bde-07b2409dd826 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:06.826140 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae5760-f332-403b-ad6a-b13987ced419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:06.827009 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae5760-f332-403b-ad6a-b13987ced419 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:06.827367 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae5760-f332-403b-ad6a-b13987ced419 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:06.827722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae5760-f332-403b-ad6a-b13987ced419 None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:06.828597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae5760-f332-403b-ad6a-b13987ced419 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:06.828977 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae5760-f332-403b-ad6a-b13987ced419 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:06.830433 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae5760-f332-403b-ad6a-b13987ced419 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:06.840023 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-e308c1f6-ce76-4354-9bde-07b2409dd826 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 12:20:06.840023 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:06.840023 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:06.840023 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:06.840023 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:06.840023 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:06.840023 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:06.840023 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:06.840023 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:06.840023 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:06.840023 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:06.840023 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:06.840023 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:20:06.840023 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:20:06.840023 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:20:06.840023 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:20:06.841235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:06.841235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:06.841235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:20:06.841235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:20:06.841235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:20:06.841235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:20:06.841235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:06.841235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:06.841235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:06.841235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:06.841235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:06.841235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:20:06.841235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:06.841235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:06.841235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:06.841235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:06.841235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:06.841235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:06.842315 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:06.842315 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:06.842315 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:20:06.842315 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:20:06.842315 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:20:06.842315 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:20:06.842315 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:20:06.842315 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:20:06.842315 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 12:20:06.842315 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:06.842881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 459/923] 10.222.0.16 () {56 vars in 1114 bytes} [Mon May 23 12:20:06 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:06.842881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae5760-f332-403b-ad6a-b13987ced419 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:06.849391 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 465/924] 10.222.0.16 () {56 vars in 1117 bytes} [Mon May 23 12:20:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:06.851992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308c1f6-ce76-4354-9bde-07b2409dd826 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:06.953067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:06.971774 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308c1f6-ce76-4354-9bde-07b2409dd826 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:06.978574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae5760-f332-403b-ad6a-b13987ced419 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:07.010722 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e10f9b-f521-45cb-8d7b-48fc06d440fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:07.011402 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e10f9b-f521-45cb-8d7b-48fc06d440fa None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:07.011689 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e10f9b-f521-45cb-8d7b-48fc06d440fa None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:07.011968 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e10f9b-f521-45cb-8d7b-48fc06d440fa None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:07.012847 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e10f9b-f521-45cb-8d7b-48fc06d440fa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:07.013165 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e10f9b-f521-45cb-8d7b-48fc06d440fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:07.013983 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:07.015100 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e10f9b-f521-45cb-8d7b-48fc06d440fa None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:07.037440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7a826-8209-40c8-bb3d-52ed331ee995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:07.038519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7a826-8209-40c8-bb3d-52ed331ee995 None admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:07.039033 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7a826-8209-40c8-bb3d-52ed331ee995 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:07.039521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7a826-8209-40c8-bb3d-52ed331ee995 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:07.040552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a7a826-8209-40c8-bb3d-52ed331ee995 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:07.043423 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a7a826-8209-40c8-bb3d-52ed331ee995 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:07.049836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 466/925] 10.222.0.16 () {56 vars in 1149 bytes} [Mon May 23 12:20:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:07.051031 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e10f9b-f521-45cb-8d7b-48fc06d440fa None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:07.078748 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 460/926] 10.222.0.16 () {60 vars in 1144 bytes} [Mon May 23 12:20:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:20:07.172471 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e10f9b-f521-45cb-8d7b-48fc06d440fa None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:07.223920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a659e-4de0-4980-8ffb-4e4bbbe3c592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:07.225200 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a659e-4de0-4980-8ffb-4e4bbbe3c592 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:07.225704 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a659e-4de0-4980-8ffb-4e4bbbe3c592 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:07.226187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a659e-4de0-4980-8ffb-4e4bbbe3c592 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:07.237918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:07.240636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcc242-f7ad-428c-a674-19bdaeae9581 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:07.240636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcc242-f7ad-428c-a674-19bdaeae9581 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:07.240636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcc242-f7ad-428c-a674-19bdaeae9581 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:07.240835 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 461/927] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:07.251099 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a659e-4de0-4980-8ffb-4e4bbbe3c592 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:07.251438 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160f4d3-08a2-4b49-81cc-14e33a219e92 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:07.251705 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160f4d3-08a2-4b49-81cc-14e33a219e92 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:07.252646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160f4d3-08a2-4b49-81cc-14e33a219e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:07.253617 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a659e-4de0-4980-8ffb-4e4bbbe3c592 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:07.273093 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-2160f4d3-08a2-4b49-81cc-14e33a219e92 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:07.274357 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-2160f4d3-08a2-4b49-81cc-14e33a219e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:07.274688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-183a659e-4de0-4980-8ffb-4e4bbbe3c592 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:20:07.274688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:07.274688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:07.274688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:07.274688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:07.274688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:07.274688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:07.274688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:07.274688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:07.274688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:07.274688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:07.274688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:07.274688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:20:07.274688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:20:07.274688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:20:07.275547 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:20:07.275547 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:07.275547 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:07.275547 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:07.275547 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:07.275547 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:07.275547 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:07.275547 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:07.275547 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:20:07.275547 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:07.275547 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:07.275547 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:07.275547 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:07.275547 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:07.275547 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:07.275547 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:07.275547 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:07.276483 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:20:07.276483 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:20:07.276483 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:20:07.276483 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:07.278063 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 467/928] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:20:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:20:07.284335 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a659e-4de0-4980-8ffb-4e4bbbe3c592 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:07.362774 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 462/929] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 12:20:07.387937 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160f4d3-08a2-4b49-81cc-14e33a219e92 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:07.415300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a659e-4de0-4980-8ffb-4e4bbbe3c592 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:07.454162 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e30468-2fef-4f4d-ad2d-5550e6747365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:07.454959 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e30468-2fef-4f4d-ad2d-5550e6747365 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:07.455248 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e30468-2fef-4f4d-ad2d-5550e6747365 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:07.455528 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e30468-2fef-4f4d-ad2d-5550e6747365 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:07.456274 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e30468-2fef-4f4d-ad2d-5550e6747365 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:07.456588 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e30468-2fef-4f4d-ad2d-5550e6747365 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:07.457999 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e30468-2fef-4f4d-ad2d-5550e6747365 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:07.470254 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 468/930] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:20:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:07.471529 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9c982-bc89-4f14-a546-eef0226f9582 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:07.471820 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9c982-bc89-4f14-a546-eef0226f9582 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:07.472119 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9c982-bc89-4f14-a546-eef0226f9582 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:07.476299 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 469/931] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:07.478225 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc9c982-bc89-4f14-a546-eef0226f9582 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:07.501821 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160f4d3-08a2-4b49-81cc-14e33a219e92 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:07.513284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:07.539994 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:07.546197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94caa64-6fe0-4678-8fad-26b0abbbbdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:07.547026 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94caa64-6fe0-4678-8fad-26b0abbbbdaf None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:07.547312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94caa64-6fe0-4678-8fad-26b0abbbbdaf None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:07.547594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94caa64-6fe0-4678-8fad-26b0abbbbdaf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:07.548292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94caa64-6fe0-4678-8fad-26b0abbbbdaf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:07.549667 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94caa64-6fe0-4678-8fad-26b0abbbbdaf None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:07.563265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-b94caa64-6fe0-4678-8fad-26b0abbbbdaf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 12:20:07.563265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:07.563265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 12:20:07.563265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 12:20:07.563265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 12:20:07.563265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 12:20:07.563265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 12:20:07.563265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:20:07.563265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 12:20:07.563265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:07.563265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:20:07.563265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:07.563265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:07.563265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:07.563265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:07.563265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:07.563265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:07.564365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:07.564365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:07.564365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:07.564365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:07.564365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:07.564365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:07.564365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 12:20:07.564365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 12:20:07.564365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 12:20:07.564365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 12:20:07.564365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:07.564365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:07.564365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:07.564365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:07.564365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:07.564365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:07.569330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:07.569330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:20:07.569330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:07.569330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:07.569330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:07.569330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:07.569330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:07.569330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:07.569330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:07.569330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:07.569330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 12:20:07.569330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:20:07.569330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 12:20:07.569330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:07.569330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 463/932] 10.222.0.16 () {56 vars in 1126 bytes} [Mon May 23 12:20:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:07.570136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9af4a4-dd21-4657-b09a-41d43c32302c None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:07.570136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9af4a4-dd21-4657-b09a-41d43c32302c None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:07.570136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9af4a4-dd21-4657-b09a-41d43c32302c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:07.585767 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-8b9af4a4-dd21-4657-b09a-41d43c32302c None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:07.587220 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-8b9af4a4-dd21-4657-b09a-41d43c32302c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:07.596765 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc9c982-bc89-4f14-a546-eef0226f9582 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:07.635177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40fe1b7-21b1-4874-b330-50ea2e2cbabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:07.636162 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fe1b7-21b1-4874-b330-50ea2e2cbabb None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:07.636471 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fe1b7-21b1-4874-b330-50ea2e2cbabb None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:07.636749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40fe1b7-21b1-4874-b330-50ea2e2cbabb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:07.644014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40fe1b7-21b1-4874-b330-50ea2e2cbabb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:07.646077 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40fe1b7-21b1-4874-b330-50ea2e2cbabb None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:07.658689 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 464/933] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:20:07.659777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9af4a4-dd21-4657-b09a-41d43c32302c None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:07.661508 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-a40fe1b7-21b1-4874-b330-50ea2e2cbabb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:20:07.661508 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:07.661508 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:07.661508 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:07.661508 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:07.661508 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:07.661508 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:07.661508 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:07.661508 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:07.661508 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:07.661508 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:07.661508 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:07.661508 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:20:07.661508 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:20:07.661508 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:20:07.662425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:20:07.662425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:07.662425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:07.662425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:07.662425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:07.662425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:07.662425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:07.662425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:07.662425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:20:07.662425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:07.662425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:07.662425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:07.662425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:07.662425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:07.662425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:07.662425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:07.662425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:07.663420 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:20:07.663420 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:20:07.663420 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:20:07.663420 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:20:07.663420 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:20:07.663420 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:20:07.663420 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:20:07.663420 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:07.663420 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 470/934] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:20:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:07.668144 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40fe1b7-21b1-4874-b330-50ea2e2cbabb None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:07.774867 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9af4a4-dd21-4657-b09a-41d43c32302c None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:07.775429 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40fe1b7-21b1-4874-b330-50ea2e2cbabb None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:07.776575 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:07.786713 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:07.814694 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adcbe06-4815-41d6-9aa9-9791586e1ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:07.816117 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adcbe06-4815-41d6-9aa9-9791586e1ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:07.816575 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adcbe06-4815-41d6-9aa9-9791586e1ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:07.818076 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adcbe06-4815-41d6-9aa9-9791586e1ab8 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:07.819248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d254b-f089-4337-9226-a7d714e34362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:07.819941 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d254b-f089-4337-9226-a7d714e34362 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:07.820256 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d254b-f089-4337-9226-a7d714e34362 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:07.820535 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d254b-f089-4337-9226-a7d714e34362 None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:07.820720 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adcbe06-4815-41d6-9aa9-9791586e1ab8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:07.821177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adcbe06-4815-41d6-9aa9-9791586e1ab8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:07.821291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d254b-f089-4337-9226-a7d714e34362 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:07.821538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d254b-f089-4337-9226-a7d714e34362 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:07.822922 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d254b-f089-4337-9226-a7d714e34362 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:07.823173 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adcbe06-4815-41d6-9aa9-9791586e1ab8 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:07.828491 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 465/935] 10.222.0.16 () {56 vars in 1122 bytes} [Mon May 23 12:20:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:07.829365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d254b-f089-4337-9226-a7d714e34362 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:07.839817 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 471/936] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:20:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:07.840923 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adcbe06-4815-41d6-9aa9-9791586e1ab8 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:07.946110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d254b-f089-4337-9226-a7d714e34362 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:07.949517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adcbe06-4815-41d6-9aa9-9791586e1ab8 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:07.983707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00421b-edd9-451f-97e3-f851f1ac524d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:07.984544 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00421b-edd9-451f-97e3-f851f1ac524d None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:07.984824 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00421b-edd9-451f-97e3-f851f1ac524d None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:07.985105 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00421b-edd9-451f-97e3-f851f1ac524d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:07.985756 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af00421b-edd9-451f-97e3-f851f1ac524d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:07.987130 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af00421b-edd9-451f-97e3-f851f1ac524d None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:07.990434 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4216376-a251-4657-96bc-dfe10c8fa5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:07.991112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4216376-a251-4657-96bc-dfe10c8fa5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:07.991398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4216376-a251-4657-96bc-dfe10c8fa5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:07.991676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4216376-a251-4657-96bc-dfe10c8fa5f9 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:07.992417 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4216376-a251-4657-96bc-dfe10c8fa5f9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:07.992728 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4216376-a251-4657-96bc-dfe10c8fa5f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:07.994076 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4216376-a251-4657-96bc-dfe10c8fa5f9 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:07.999239 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 472/937] 10.222.0.16 () {56 vars in 1122 bytes} [Mon May 23 12:20:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:08.000000 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4216376-a251-4657-96bc-dfe10c8fa5f9 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:08.001721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-af00421b-edd9-451f-97e3-f851f1ac524d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 12:20:08.001721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:08.001721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 12:20:08.001721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 12:20:08.001721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 12:20:08.001721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 12:20:08.001721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 12:20:08.001721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:20:08.001721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 12:20:08.001721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:08.001721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:20:08.001721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:08.001721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:08.001721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:08.001721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:08.001721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:08.002668 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:08.002668 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:08.002668 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:08.002668 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:08.002668 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:08.002668 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:08.002668 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:08.002668 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 12:20:08.002668 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 12:20:08.002668 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 12:20:08.002668 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 12:20:08.002668 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:08.002668 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:08.002668 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:08.002668 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:08.002668 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:08.003802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:08.003802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:08.003802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:20:08.003802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:08.003802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:08.003802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:08.003802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:08.003802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:08.003802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:08.003802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:08.003802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:08.003802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 12:20:08.003802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:20:08.003802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 12:20:08.003802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:08.008710 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 466/938] 10.222.0.16 () {56 vars in 1132 bytes} [Mon May 23 12:20:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:08.009568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00421b-edd9-451f-97e3-f851f1ac524d None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:08.048048 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:08.048278 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:08.121893 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4216376-a251-4657-96bc-dfe10c8fa5f9 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:08.124778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00421b-edd9-451f-97e3-f851f1ac524d None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:08.159220 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5965d4-abb3-47f3-aa1e-baa600609e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:08.159420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7659b7-0308-4998-8bb5-f5daa4e1120a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:08.160031 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7659b7-0308-4998-8bb5-f5daa4e1120a None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:08.160377 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7659b7-0308-4998-8bb5-f5daa4e1120a None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:08.160708 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7659b7-0308-4998-8bb5-f5daa4e1120a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:08.160997 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5965d4-abb3-47f3-aa1e-baa600609e0e None admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:08.161507 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7659b7-0308-4998-8bb5-f5daa4e1120a None admin] RBAC: Adding query filter params (interface=public, service_id=ad456971d51f440ea92e0c02b48e6f92, region_id=RegionOne) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:08.161765 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5965d4-abb3-47f3-aa1e-baa600609e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:08.161888 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7659b7-0308-4998-8bb5-f5daa4e1120a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:08.162298 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5965d4-abb3-47f3-aa1e-baa600609e0e None admin] PATH_INFO: `/v3/projects/839585501f414e1ab02501422510e54f/users/733f8296ec3d4795a1140c1e1cb6eb95/roles/3d40411f939f4ff6af2e78c756f6cc9e` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:08.163224 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7659b7-0308-4998-8bb5-f5daa4e1120a None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:08.164994 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5965d4-abb3-47f3-aa1e-baa600609e0e None admin] RBAC: Authorizing `identity:create_grant(project_id=839585501f414e1ab02501422510e54f, user_id=733f8296ec3d4795a1140c1e1cb6eb95, role_id=3d40411f939f4ff6af2e78c756f6cc9e)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:08.166557 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5965d4-abb3-47f3-aa1e-baa600609e0e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:08.168521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 467/939] 10.222.0.16 () {56 vars in 1251 bytes} [Mon May 23 12:20:08 2022] GET /identity/v3/endpoints?service_id=ad456971d51f440ea92e0c02b48e6f92&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:08.169424 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7659b7-0308-4998-8bb5-f5daa4e1120a None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:08.187468 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 473/940] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 12:20:07 2022] PUT /identity/v3/projects/839585501f414e1ab02501422510e54f/users/733f8296ec3d4795a1140c1e1cb6eb95/roles/3d40411f939f4ff6af2e78c756f6cc9e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:20:08.188111 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5965d4-abb3-47f3-aa1e-baa600609e0e None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:08.300368 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:08.309014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5965d4-abb3-47f3-aa1e-baa600609e0e None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:08.326179 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:08.337926 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7659b7-0308-4998-8bb5-f5daa4e1120a None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:08.352202 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a833f-ca76-4556-b756-117e7627d109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:08.352916 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a833f-ca76-4556-b756-117e7627d109 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:08.353197 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a833f-ca76-4556-b756-117e7627d109 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:08.353485 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a833f-ca76-4556-b756-117e7627d109 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:08.354262 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63a833f-ca76-4556-b756-117e7627d109 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:08.355553 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63a833f-ca76-4556-b756-117e7627d109 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:08.362703 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 474/941] 10.222.0.16 () {56 vars in 1087 bytes} [Mon May 23 12:20:08 2022] GET /identity/v3/services => generated 1497 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:20:08.387259 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48f23a-7a97-427e-b589-3e057f8f6ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:08.388027 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48f23a-7a97-427e-b589-3e057f8f6ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:08.388353 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48f23a-7a97-427e-b589-3e057f8f6ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:08.388636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48f23a-7a97-427e-b589-3e057f8f6ef9 None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:08.389424 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e48f23a-7a97-427e-b589-3e057f8f6ef9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:08.389738 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e48f23a-7a97-427e-b589-3e057f8f6ef9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:08.391209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e48f23a-7a97-427e-b589-3e057f8f6ef9 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:08.397507 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 468/942] 10.222.0.16 () {56 vars in 1126 bytes} [Mon May 23 12:20:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:08.403687 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a833f-ca76-4556-b756-117e7627d109 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:08.524845 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a833f-ca76-4556-b756-117e7627d109 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:08.562142 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b96a8d-28a7-42e5-a92b-5b061e608d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:08.562735 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b96a8d-28a7-42e5-a92b-5b061e608d3f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:08.563020 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b96a8d-28a7-42e5-a92b-5b061e608d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:08.563360 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b96a8d-28a7-42e5-a92b-5b061e608d3f None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:08.564083 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b96a8d-28a7-42e5-a92b-5b061e608d3f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:08.564470 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b96a8d-28a7-42e5-a92b-5b061e608d3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:08.565971 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b96a8d-28a7-42e5-a92b-5b061e608d3f None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:08.571964 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 475/943] 10.222.0.16 () {56 vars in 1126 bytes} [Mon May 23 12:20:08 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:08.577873 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48f23a-7a97-427e-b589-3e057f8f6ef9 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:08.683056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48f23a-7a97-427e-b589-3e057f8f6ef9 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:08.720608 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb3977-e674-46a1-8b98-d9a4f68d70ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:08.721673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb3977-e674-46a1-8b98-d9a4f68d70ed None admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:08.722167 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb3977-e674-46a1-8b98-d9a4f68d70ed None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:08.722652 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb3977-e674-46a1-8b98-d9a4f68d70ed None admin] PATH_INFO: `/v3/endpoints` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:08.724333 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eb3977-e674-46a1-8b98-d9a4f68d70ed None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:08.725953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eb3977-e674-46a1-8b98-d9a4f68d70ed None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:08.752273 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 469/944] 10.222.0.16 () {60 vars in 1144 bytes} [Mon May 23 12:20:08 2022] POST /identity/v3/endpoints => generated 347 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:20:09.427690 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503e813-0292-4c3d-9fcb-a7e274b9d641 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:09.428326 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503e813-0292-4c3d-9fcb-a7e274b9d641 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:09.428326 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503e813-0292-4c3d-9fcb-a7e274b9d641 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:09.429870 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 476/945] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:09.435562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ef2fc-7989-492b-af85-0cee5a4e72bb None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:09.435991 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ef2fc-7989-492b-af85-0cee5a4e72bb None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:09.436188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ef2fc-7989-492b-af85-0cee5a4e72bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:09.454643 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-b06ef2fc-7989-492b-af85-0cee5a4e72bb None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:09.456279 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-b06ef2fc-7989-492b-af85-0cee5a4e72bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:09.467604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:09.539974 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 470/946] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:20:09.564521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503e813-0292-4c3d-9fcb-a7e274b9d641 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:09.591042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:09.673209 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503e813-0292-4c3d-9fcb-a7e274b9d641 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:09.708727 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d18b3-d64f-4f7f-8ab9-6894b0a93f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:09.709737 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d18b3-d64f-4f7f-8ab9-6894b0a93f21 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:09.710222 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d18b3-d64f-4f7f-8ab9-6894b0a93f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:09.710717 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d18b3-d64f-4f7f-8ab9-6894b0a93f21 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:09.711673 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d18b3-d64f-4f7f-8ab9-6894b0a93f21 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:09.713376 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d18b3-d64f-4f7f-8ab9-6894b0a93f21 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:09.714941 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: WARNING keystone.server.flask.common [None req-b92d18b3-d64f-4f7f-8ab9-6894b0a93f21 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 23 12:20:09.732856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.sql.core [None req-b92d18b3-d64f-4f7f-8ab9-6894b0a93f21 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 23 12:20:09.732856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 23 12:20:09.732856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [parameters: {'id': '362b67575c2f45cc9198de0993cc6543', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 23 12:20:09.732856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80215) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 23 12:20:09.750969 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-b92d18b3-d64f-4f7f-8ab9-6894b0a93f21 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 362b67575c2f45cc9198de0993cc6543.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 362b67575c2f45cc9198de0993cc6543. May 23 12:20:09.750969 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:09.750969 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 12:20:09.750969 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 12:20:09.750969 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 12:20:09.750969 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 12:20:09.750969 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 12:20:09.750969 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application result = self._query(query) May 23 12:20:09.750969 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 12:20:09.750969 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application conn.query(q) May 23 12:20:09.750969 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 12:20:09.750969 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 12:20:09.750969 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 12:20:09.751837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application result.read() May 23 12:20:09.751837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 12:20:09.751837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 12:20:09.751837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 12:20:09.751837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 12:20:09.751837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 12:20:09.751837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 12:20:09.751837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 12:20:09.751837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 12:20:09.751837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 23 12:20:09.751837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:09.751837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 23 12:20:09.751837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:09.751837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:09.751837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 23 12:20:09.751837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 23 12:20:09.751837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 23 12:20:09.751837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return project_ref.to_dict() May 23 12:20:09.752842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 12:20:09.752842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application next(self.gen) May 23 12:20:09.752842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 23 12:20:09.752842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application yield resource May 23 12:20:09.752842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 12:20:09.752842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application next(self.gen) May 23 12:20:09.752842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 23 12:20:09.752842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self.session.rollback() May 23 12:20:09.752842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 23 12:20:09.752842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self.force_reraise() May 23 12:20:09.752842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 23 12:20:09.752842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise self.value May 23 12:20:09.752842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 23 12:20:09.752842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 23 12:20:09.752842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 23 12:20:09.752842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application session.commit() May 23 12:20:09.752842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 23 12:20:09.753772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 23 12:20:09.753772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 23 12:20:09.753772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._prepare_impl() May 23 12:20:09.753772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 23 12:20:09.753772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self.session.flush() May 23 12:20:09.753772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 23 12:20:09.753772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._flush(objects) May 23 12:20:09.753772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 23 12:20:09.753772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 23 12:20:09.753772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 23 12:20:09.753772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application compat.raise_( May 23 12:20:09.753772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 12:20:09.753772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception May 23 12:20:09.753772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 23 12:20:09.753772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application flush_context.execute() May 23 12:20:09.753772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 23 12:20:09.753772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rec.execute(self) May 23 12:20:09.755254 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 23 12:20:09.755254 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 23 12:20:09.755254 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 23 12:20:09.755254 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application _emit_insert_statements( May 23 12:20:09.755254 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 23 12:20:09.755254 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application c = connection._execute_20( May 23 12:20:09.755254 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 23 12:20:09.755254 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 23 12:20:09.755254 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 23 12:20:09.755254 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 23 12:20:09.755254 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 23 12:20:09.755254 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ret = self._execute_context( May 23 12:20:09.755254 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 23 12:20:09.755254 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 23 12:20:09.755254 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 23 12:20:09.756043 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 23 12:20:09.756043 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 12:20:09.756043 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception May 23 12:20:09.756043 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 12:20:09.756043 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 12:20:09.756043 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 12:20:09.756043 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 12:20:09.756043 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 12:20:09.756043 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application result = self._query(query) May 23 12:20:09.756043 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 12:20:09.756043 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application conn.query(q) May 23 12:20:09.756043 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 12:20:09.756043 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 12:20:09.756043 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 12:20:09.756043 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application result.read() May 23 12:20:09.756043 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 12:20:09.756043 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 12:20:09.758888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 12:20:09.758888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 12:20:09.758888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 12:20:09.758888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 12:20:09.758888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 12:20:09.758888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 12:20:09.758888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 23 12:20:09.758888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 23 12:20:09.758888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [parameters: {'id': '362b67575c2f45cc9198de0993cc6543', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 23 12:20:09.758888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 23 12:20:09.758888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:09.758888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:20:09.758888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:09.758888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:09.758888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 23 12:20:09.760839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 23 12:20:09.760839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 23 12:20:09.760839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 23 12:20:09.760839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 23 12:20:09.760839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:09.760839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:20:09.760839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:09.760839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:09.760839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:09.760839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:09.760839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:09.760839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:09.760839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:09.760839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:09.760839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:09.760839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:09.760839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:09.763037 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:09.763037 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 23 12:20:09.763037 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 23 12:20:09.763037 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:09.763037 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:09.763037 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 23 12:20:09.763037 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.Conflict( May 23 12:20:09.763037 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 362b67575c2f45cc9198de0993cc6543. May 23 12:20:09.763037 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:09.765661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 477/947] 10.222.0.16 () {60 vars in 1140 bytes} [Mon May 23 12:20:09 2022] POST /identity/v3/projects => generated 262 bytes in 201 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:09.773254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ef2fc-7989-492b-af85-0cee5a4e72bb None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:09.877062 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ef2fc-7989-492b-af85-0cee5a4e72bb None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:09.912986 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce448df-866e-4d03-8c1a-dc13d8690263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:09.913782 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce448df-866e-4d03-8c1a-dc13d8690263 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:09.914070 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce448df-866e-4d03-8c1a-dc13d8690263 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:09.914348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce448df-866e-4d03-8c1a-dc13d8690263 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:09.921420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce448df-866e-4d03-8c1a-dc13d8690263 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:09.922884 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce448df-866e-4d03-8c1a-dc13d8690263 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:09.925471 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:09.939374 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-6ce448df-866e-4d03-8c1a-dc13d8690263 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:20:09.939374 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:09.939374 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:09.939374 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:09.939374 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:09.939374 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:09.939374 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:09.939374 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:09.939374 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:09.939374 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:09.939374 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:09.939374 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:09.939374 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:20:09.939374 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:20:09.939374 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:20:09.940341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:20:09.940341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:09.940341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:09.940341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:09.940341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:09.940341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:09.940341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:09.940341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:09.940341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:20:09.940341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:09.940341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:09.940341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:09.940341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:09.940341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:09.940341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:09.940341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:09.940341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:09.941294 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:20:09.941294 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:20:09.941294 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:20:09.941294 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:20:09.941294 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:20:09.941294 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:20:09.941294 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:20:09.941294 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:09.941294 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 471/948] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:20:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:09.949066 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d18b3-d64f-4f7f-8ab9-6894b0a93f21 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:10.037103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:10.060206 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d18b3-d64f-4f7f-8ab9-6894b0a93f21 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:10.095977 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6785d9-a9fc-4b4c-b675-d4e9155a741f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:10.096834 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6785d9-a9fc-4b4c-b675-d4e9155a741f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:10.097221 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6785d9-a9fc-4b4c-b675-d4e9155a741f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:10.097599 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6785d9-a9fc-4b4c-b675-d4e9155a741f None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:10.098990 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6785d9-a9fc-4b4c-b675-d4e9155a741f None admin] RBAC: Adding query filter params (name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:10.099399 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6785d9-a9fc-4b4c-b675-d4e9155a741f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:10.101156 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6785d9-a9fc-4b4c-b675-d4e9155a741f None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:10.128307 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 478/949] 10.222.0.16 () {56 vars in 1112 bytes} [Mon May 23 12:20:09 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:10.577956 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a41ab-53f6-41cb-8d64-86668fbfc379 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:10.578988 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a41ab-53f6-41cb-8d64-86668fbfc379 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:10.579496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a41ab-53f6-41cb-8d64-86668fbfc379 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:10.581356 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 472/950] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:10.587320 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a183fe8-1b56-48e9-8024-9b3c0eaf78b7 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:10.588126 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a183fe8-1b56-48e9-8024-9b3c0eaf78b7 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:10.588648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a183fe8-1b56-48e9-8024-9b3c0eaf78b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:10.606827 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-4a183fe8-1b56-48e9-8024-9b3c0eaf78b7 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:10.608683 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-4a183fe8-1b56-48e9-8024-9b3c0eaf78b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:10.675539 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8659eed-4ec6-4103-981c-53abbec4c7b0 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:10.675851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8659eed-4ec6-4103-981c-53abbec4c7b0 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:10.676155 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8659eed-4ec6-4103-981c-53abbec4c7b0 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:10.677924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 473/951] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:10.683058 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf24509-61ad-4a7a-b4f8-50f90ee8f486 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:10.683353 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf24509-61ad-4a7a-b4f8-50f90ee8f486 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:10.683522 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 479/952] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:20:10.683620 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf24509-61ad-4a7a-b4f8-50f90ee8f486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:10.704038 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-5bf24509-61ad-4a7a-b4f8-50f90ee8f486 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:10.705304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-5bf24509-61ad-4a7a-b4f8-50f90ee8f486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:10.706993 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a183fe8-1b56-48e9-8024-9b3c0eaf78b7 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:10.782970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 474/953] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:20:10.793122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:10.806616 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf24509-61ad-4a7a-b4f8-50f90ee8f486 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:10.835062 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a183fe8-1b56-48e9-8024-9b3c0eaf78b7 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:10.880231 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c90a86-57f3-4120-9868-73ba846afa3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:10.881399 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c90a86-57f3-4120-9868-73ba846afa3f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:10.881782 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c90a86-57f3-4120-9868-73ba846afa3f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:10.882160 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c90a86-57f3-4120-9868-73ba846afa3f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:10.893216 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c90a86-57f3-4120-9868-73ba846afa3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:20:10.909163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-a3c90a86-57f3-4120-9868-73ba846afa3f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 12:20:10.909163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:10.909163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:10.909163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:10.909163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:10.909163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:10.909163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:10.909163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:10.909163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:10.909163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:10.909163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:10.909163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:10.909163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:20:10.909163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:20:10.909163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:20:10.909163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:20:10.910176 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:20:10.910176 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:20:10.910176 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:20:10.910176 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:20:10.910176 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:10.910176 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:10.910176 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:10.910176 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:10.910176 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:10.910176 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:10.910176 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:10.910176 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:20:10.910176 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:10.910176 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:10.910176 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:10.910176 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:10.911558 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:10.911558 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:10.911558 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:10.911558 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:10.911558 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:20:10.911558 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:20:10.911558 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:20:10.911558 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:20:10.911558 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:20:10.911558 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:20:10.911558 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 12:20:10.911558 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:10.912268 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 480/954] 10.222.0.16 () {56 vars in 1120 bytes} [Mon May 23 12:20:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:20:10.917993 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c90a86-57f3-4120-9868-73ba846afa3f None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:10.918957 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf24509-61ad-4a7a-b4f8-50f90ee8f486 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:10.962712 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:10.970104 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0588c50-70ac-47b1-8c74-734c53dd5c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:10.970914 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0588c50-70ac-47b1-8c74-734c53dd5c54 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:10.971193 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0588c50-70ac-47b1-8c74-734c53dd5c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:10.971468 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0588c50-70ac-47b1-8c74-734c53dd5c54 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:10.972273 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0588c50-70ac-47b1-8c74-734c53dd5c54 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:10.973977 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0588c50-70ac-47b1-8c74-734c53dd5c54 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:10.987111 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-f0588c50-70ac-47b1-8c74-734c53dd5c54 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 12:20:10.987111 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:10.987111 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 12:20:10.987111 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 12:20:10.987111 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 12:20:10.987111 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 12:20:10.987111 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 12:20:10.987111 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:20:10.987111 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 12:20:10.987111 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:10.987111 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:20:10.987111 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:10.987111 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:10.987111 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:10.987111 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:10.987111 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:10.987111 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:10.988257 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:10.988257 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:10.988257 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:10.988257 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:10.988257 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:10.988257 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:10.988257 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 12:20:10.988257 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 12:20:10.988257 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 12:20:10.988257 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 12:20:10.988257 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:10.988257 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:10.988257 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:10.988257 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:10.988257 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:10.988257 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:10.989350 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:10.989350 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:20:10.989350 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:10.989350 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:10.989350 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:10.989350 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:10.989350 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:10.989350 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:10.989350 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:10.989350 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:10.989350 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 12:20:10.989350 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:20:10.989350 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 12:20:10.989350 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:10.989350 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 475/955] 10.222.0.16 () {56 vars in 1126 bytes} [Mon May 23 12:20:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:20:10.994489 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0588c50-70ac-47b1-8c74-734c53dd5c54 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:11.058267 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c90a86-57f3-4120-9868-73ba846afa3f None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:11.087455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:11.096802 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bcab4-7879-4de9-af58-ee9d87c31e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:11.097773 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bcab4-7879-4de9-af58-ee9d87c31e8b None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:11.098279 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bcab4-7879-4de9-af58-ee9d87c31e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:11.098773 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bcab4-7879-4de9-af58-ee9d87c31e8b None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:11.099750 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0bcab4-7879-4de9-af58-ee9d87c31e8b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:11.100274 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0bcab4-7879-4de9-af58-ee9d87c31e8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:11.101834 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0bcab4-7879-4de9-af58-ee9d87c31e8b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:11.106350 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0588c50-70ac-47b1-8c74-734c53dd5c54 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:11.116246 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 481/956] 10.222.0.16 () {56 vars in 1115 bytes} [Mon May 23 12:20:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:11.117447 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bcab4-7879-4de9-af58-ee9d87c31e8b None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:11.143462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf6d5f-c75f-481d-ab36-ea8d0866a240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:11.144112 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf6d5f-c75f-481d-ab36-ea8d0866a240 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:11.144432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf6d5f-c75f-481d-ab36-ea8d0866a240 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:11.144706 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf6d5f-c75f-481d-ab36-ea8d0866a240 None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:11.145725 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edf6d5f-c75f-481d-ab36-ea8d0866a240 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:11.146035 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edf6d5f-c75f-481d-ab36-ea8d0866a240 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:11.147409 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edf6d5f-c75f-481d-ab36-ea8d0866a240 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:11.152608 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 476/957] 10.222.0.16 () {56 vars in 1122 bytes} [Mon May 23 12:20:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:11.153806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315966e6-cf68-453a-a6c8-d2fd833804a7 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:11.154125 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315966e6-cf68-453a-a6c8-d2fd833804a7 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:11.154370 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315966e6-cf68-453a-a6c8-d2fd833804a7 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:11.156222 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 477/958] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:11.158178 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315966e6-cf68-453a-a6c8-d2fd833804a7 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:11.220434 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:11.233193 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bcab4-7879-4de9-af58-ee9d87c31e8b None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:11.265286 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315966e6-cf68-453a-a6c8-d2fd833804a7 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:11.270879 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd125eb0-d470-4adf-bceb-3154011e3e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:11.271683 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd125eb0-d470-4adf-bceb-3154011e3e1a None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:11.271967 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd125eb0-d470-4adf-bceb-3154011e3e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:11.272271 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd125eb0-d470-4adf-bceb-3154011e3e1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:11.283943 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd125eb0-d470-4adf-bceb-3154011e3e1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:11.285622 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd125eb0-d470-4adf-bceb-3154011e3e1a None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:11.301670 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c2ff4-c0df-4b37-a5cd-fb1943a25548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:11.302337 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c2ff4-c0df-4b37-a5cd-fb1943a25548 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:11.302626 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c2ff4-c0df-4b37-a5cd-fb1943a25548 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:11.302906 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c2ff4-c0df-4b37-a5cd-fb1943a25548 None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:11.303619 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c2ff4-c0df-4b37-a5cd-fb1943a25548 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:11.303928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c2ff4-c0df-4b37-a5cd-fb1943a25548 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:11.304577 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-dd125eb0-d470-4adf-bceb-3154011e3e1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:20:11.304577 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:11.304577 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:11.304577 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:11.304577 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:11.304577 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:11.304577 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:11.304577 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:11.304577 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:11.304577 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:11.304577 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:11.304577 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:11.304577 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:20:11.304577 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:20:11.304577 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:20:11.305428 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:20:11.305428 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:11.305428 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:11.305428 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:11.305428 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:11.305428 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:11.305428 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:11.305428 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:11.305428 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:20:11.305428 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:11.305428 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:11.305428 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:11.305428 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:11.305428 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:11.305428 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:11.305428 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:11.305428 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:11.311286 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:20:11.311286 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:20:11.311286 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:20:11.311286 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:11.311538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c2ff4-c0df-4b37-a5cd-fb1943a25548 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:11.311620 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 482/959] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:20:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:11.311620 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299a43a-d99d-4a55-b1de-8b59d258a787 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:11.311620 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299a43a-d99d-4a55-b1de-8b59d258a787 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:11.311620 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299a43a-d99d-4a55-b1de-8b59d258a787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:11.316205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 478/960] 10.222.0.16 () {56 vars in 1122 bytes} [Mon May 23 12:20:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:11.318081 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c2ff4-c0df-4b37-a5cd-fb1943a25548 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:11.332103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-6299a43a-d99d-4a55-b1de-8b59d258a787 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:11.334483 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-6299a43a-d99d-4a55-b1de-8b59d258a787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:11.372867 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:11.416678 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 483/961] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:20:11.418003 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6299a43a-d99d-4a55-b1de-8b59d258a787 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:11.428508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c2ff4-c0df-4b37-a5cd-fb1943a25548 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:11.453099 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:11.467496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a2293-e720-4663-9d19-da1b079a5598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:11.468162 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a2293-e720-4663-9d19-da1b079a5598 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:11.468491 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a2293-e720-4663-9d19-da1b079a5598 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:11.468771 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a2293-e720-4663-9d19-da1b079a5598 None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:11.469502 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452a2293-e720-4663-9d19-da1b079a5598 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:11.469813 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452a2293-e720-4663-9d19-da1b079a5598 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:11.471196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452a2293-e720-4663-9d19-da1b079a5598 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:11.484523 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 479/962] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:11.490177 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a2293-e720-4663-9d19-da1b079a5598 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:11.533721 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6299a43a-d99d-4a55-b1de-8b59d258a787 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:11.570189 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf79762-d0e7-49b5-8f2f-06377ba5c14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:11.571217 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf79762-d0e7-49b5-8f2f-06377ba5c14a None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:11.571513 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf79762-d0e7-49b5-8f2f-06377ba5c14a None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:11.571797 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf79762-d0e7-49b5-8f2f-06377ba5c14a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:11.572593 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf79762-d0e7-49b5-8f2f-06377ba5c14a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:11.574295 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf79762-d0e7-49b5-8f2f-06377ba5c14a None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:11.596226 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 484/963] 10.222.0.16 () {60 vars in 1144 bytes} [Mon May 23 12:20:11 2022] POST /identity/v3/endpoints => generated 349 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:20:11.596226 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf79762-d0e7-49b5-8f2f-06377ba5c14a None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:11.608483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a2293-e720-4663-9d19-da1b079a5598 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:11.631989 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:11.676915 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6c1ba-528e-45cf-be16-7b5ef80a0388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:11.678879 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6c1ba-528e-45cf-be16-7b5ef80a0388 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:11.679424 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6c1ba-528e-45cf-be16-7b5ef80a0388 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:11.679912 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6c1ba-528e-45cf-be16-7b5ef80a0388 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:11.681271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6c1ba-528e-45cf-be16-7b5ef80a0388 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:11.683635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6c1ba-528e-45cf-be16-7b5ef80a0388 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:11.702030 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-48c6c1ba-528e-45cf-be16-7b5ef80a0388 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 23 12:20:11.702030 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:11.702030 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 12:20:11.702030 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 12:20:11.702030 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 12:20:11.702030 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 12:20:11.702030 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 12:20:11.702030 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:20:11.702030 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 12:20:11.702030 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:11.702030 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:20:11.702030 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:11.702030 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:11.702030 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:11.702030 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:11.702030 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:11.702030 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:11.703219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:11.703219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:11.703219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:11.703219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:11.703219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:11.703219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:11.703219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 12:20:11.703219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 12:20:11.703219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 12:20:11.703219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 12:20:11.703219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:11.703219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:11.703219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:11.703219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:11.703219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:11.703219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:11.704113 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:11.704113 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:20:11.704113 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:11.704113 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:11.704113 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:11.704113 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:11.704113 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:11.704113 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:11.704113 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:11.704113 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:11.704113 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 12:20:11.704113 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:20:11.704113 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 12:20:11.704113 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:11.710928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 480/964] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:20:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:11.711856 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6c1ba-528e-45cf-be16-7b5ef80a0388 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:11.737923 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf79762-d0e7-49b5-8f2f-06377ba5c14a None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:11.762271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:11.791878 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c88c1-4d47-4af0-87ac-47a44b880a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:11.792750 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c88c1-4d47-4af0-87ac-47a44b880a05 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:11.793040 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c88c1-4d47-4af0-87ac-47a44b880a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:11.793317 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c88c1-4d47-4af0-87ac-47a44b880a05 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:11.806563 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c88c1-4d47-4af0-87ac-47a44b880a05 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:11.808016 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6c88c1-4d47-4af0-87ac-47a44b880a05 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:11.825779 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-dc6c88c1-4d47-4af0-87ac-47a44b880a05 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 12:20:11.825779 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:11.825779 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:11.825779 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:11.825779 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:11.825779 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:11.825779 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:11.825779 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:11.825779 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:11.825779 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:11.825779 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:11.825779 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:11.825779 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:20:11.825779 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:20:11.825779 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:20:11.825779 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:20:11.826804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:11.826804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:11.826804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:20:11.826804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:20:11.826804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:20:11.826804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:20:11.826804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:11.826804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:11.826804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:11.826804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:11.826804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:11.826804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:20:11.826804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:11.826804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:11.826804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:11.826804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:11.826804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:11.826804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:11.827774 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:11.827774 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:11.827774 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:20:11.827774 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:20:11.827774 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:20:11.827774 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:20:11.827774 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:20:11.827774 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:20:11.827774 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 12:20:11.827774 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:11.827774 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 485/965] 10.222.0.16 () {56 vars in 1117 bytes} [Mon May 23 12:20:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:11.835187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c88c1-4d47-4af0-87ac-47a44b880a05 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:11.843518 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6c1ba-528e-45cf-be16-7b5ef80a0388 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:11.892297 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6732ac8e-f220-4821-ae5c-c2ac85d403b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:11.893152 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6732ac8e-f220-4821-ae5c-c2ac85d403b5 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:11.893826 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6732ac8e-f220-4821-ae5c-c2ac85d403b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:11.894115 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6732ac8e-f220-4821-ae5c-c2ac85d403b5 None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:11.894861 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6732ac8e-f220-4821-ae5c-c2ac85d403b5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:11.895179 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6732ac8e-f220-4821-ae5c-c2ac85d403b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:11.896671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6732ac8e-f220-4821-ae5c-c2ac85d403b5 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:11.908271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 481/966] 10.222.0.16 () {56 vars in 1112 bytes} [Mon May 23 12:20:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:11.909699 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6732ac8e-f220-4821-ae5c-c2ac85d403b5 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:11.943028 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:11.958036 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c88c1-4d47-4af0-87ac-47a44b880a05 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:12.003736 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99344b8b-3da9-49a8-a430-94ca1bbb6fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:12.005175 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99344b8b-3da9-49a8-a430-94ca1bbb6fba None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:12.005479 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99344b8b-3da9-49a8-a430-94ca1bbb6fba None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:12.005964 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99344b8b-3da9-49a8-a430-94ca1bbb6fba None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:12.006827 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99344b8b-3da9-49a8-a430-94ca1bbb6fba None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:12.007959 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99344b8b-3da9-49a8-a430-94ca1bbb6fba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:12.009074 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99344b8b-3da9-49a8-a430-94ca1bbb6fba None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:12.040614 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 486/967] 10.222.0.16 () {56 vars in 1149 bytes} [Mon May 23 12:20:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:12.042740 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99344b8b-3da9-49a8-a430-94ca1bbb6fba None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:12.069537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6732ac8e-f220-4821-ae5c-c2ac85d403b5 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:12.077521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:12.107954 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c0940-4821-41d0-b480-3e5ff8fd3eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:12.108835 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c0940-4821-41d0-b480-3e5ff8fd3eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:12.109226 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c0940-4821-41d0-b480-3e5ff8fd3eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:12.109608 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c0940-4821-41d0-b480-3e5ff8fd3eb5 None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:12.110426 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c0940-4821-41d0-b480-3e5ff8fd3eb5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:12.110837 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c0940-4821-41d0-b480-3e5ff8fd3eb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:12.112451 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c0940-4821-41d0-b480-3e5ff8fd3eb5 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:12.124224 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 482/968] 10.222.0.16 () {56 vars in 1112 bytes} [Mon May 23 12:20:11 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:12.179049 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99344b8b-3da9-49a8-a430-94ca1bbb6fba None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:12.217672 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed4a96-a485-4cd3-8767-687766662547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:12.218541 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed4a96-a485-4cd3-8767-687766662547 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:12.218925 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed4a96-a485-4cd3-8767-687766662547 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:12.219307 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed4a96-a485-4cd3-8767-687766662547 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:12.237576 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:12.305016 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ed4a96-a485-4cd3-8767-687766662547 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:12.306821 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ed4a96-a485-4cd3-8767-687766662547 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:12.421299 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 487/969] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:20:12 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 12:20:12.427267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c0940-4821-41d0-b480-3e5ff8fd3eb5 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:12.462824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa40a1-6ac4-4984-95a9-3f7f1f6d0fbe None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:12.463029 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa40a1-6ac4-4984-95a9-3f7f1f6d0fbe None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:12.463306 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa40a1-6ac4-4984-95a9-3f7f1f6d0fbe None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:12.465374 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 488/970] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:20:12.473842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d4876-6745-4931-bec2-96158a52a6a5 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:12.474382 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d4876-6745-4931-bec2-96158a52a6a5 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:12.474736 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d4876-6745-4931-bec2-96158a52a6a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:12.496232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-f44d4876-6745-4931-bec2-96158a52a6a5 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:12.497983 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-f44d4876-6745-4931-bec2-96158a52a6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:12.544538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c0940-4821-41d0-b480-3e5ff8fd3eb5 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:12.559557 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:12.582955 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c0f3f-30d4-42b1-92b5-dbaa6452eaf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:12.583984 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c0f3f-30d4-42b1-92b5-dbaa6452eaf5 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:12.584466 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c0f3f-30d4-42b1-92b5-dbaa6452eaf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:12.584887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c0f3f-30d4-42b1-92b5-dbaa6452eaf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:12.592218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 489/971] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:20:12.603723 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c0f3f-30d4-42b1-92b5-dbaa6452eaf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:12.605486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c0f3f-30d4-42b1-92b5-dbaa6452eaf5 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:12.619810 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d4876-6745-4931-bec2-96158a52a6a5 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:12.628836 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-e66c0f3f-30d4-42b1-92b5-dbaa6452eaf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:20:12.628836 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:12.628836 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:12.628836 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:12.628836 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:12.628836 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:12.628836 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:12.628836 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:12.628836 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:12.628836 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:12.628836 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:12.628836 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:12.628836 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:20:12.628836 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:20:12.628836 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:20:12.630195 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:20:12.630195 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:12.630195 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:12.630195 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:12.630195 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:12.630195 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:12.630195 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:12.630195 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:12.630195 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:20:12.630195 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:12.630195 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:12.630195 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:12.630195 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:12.630195 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:12.630195 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:12.630195 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:12.630195 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:12.631127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:20:12.631127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:20:12.631127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:20:12.631127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:12.633494 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 483/972] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:20:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:12.640192 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c0f3f-30d4-42b1-92b5-dbaa6452eaf5 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:12.677277 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:12.737211 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d4876-6745-4931-bec2-96158a52a6a5 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:12.775501 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef62644-46c9-41f2-bb86-43ecb787e920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:12.776559 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef62644-46c9-41f2-bb86-43ecb787e920 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:12.777065 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef62644-46c9-41f2-bb86-43ecb787e920 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:12.777542 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef62644-46c9-41f2-bb86-43ecb787e920 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:12.778438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef62644-46c9-41f2-bb86-43ecb787e920 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:12.780162 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef62644-46c9-41f2-bb86-43ecb787e920 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:12.787364 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c0f3f-30d4-42b1-92b5-dbaa6452eaf5 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:12.796239 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.sql.core [None req-4ef62644-46c9-41f2-bb86-43ecb787e920 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 23 12:20:12.796239 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 23 12:20:12.796239 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [parameters: {'user_id': 'c2ee80cc85f64f0f92f8a094c00208d8', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 23 12:20:12.796239 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80215) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 23 12:20:12.805932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-4ef62644-46c9-41f2-bb86-43ecb787e920 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 23 12:20:12.805932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:12.805932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 12:20:12.805932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 12:20:12.805932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 12:20:12.805932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 12:20:12.805932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 12:20:12.805932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application result = self._query(query) May 23 12:20:12.805932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 12:20:12.805932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application conn.query(q) May 23 12:20:12.805932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 12:20:12.805932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 12:20:12.805932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 12:20:12.805932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application result.read() May 23 12:20:12.805932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 12:20:12.806957 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 12:20:12.806957 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 12:20:12.806957 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 12:20:12.806957 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 12:20:12.806957 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 12:20:12.806957 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 12:20:12.806957 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 12:20:12.806957 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 23 12:20:12.806957 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:12.806957 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 23 12:20:12.806957 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:12.806957 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:12.806957 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 23 12:20:12.806957 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 23 12:20:12.806957 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 23 12:20:12.806957 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 23 12:20:12.806957 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 12:20:12.806957 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application next(self.gen) May 23 12:20:12.807992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 23 12:20:12.807992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application yield resource May 23 12:20:12.807992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 12:20:12.807992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application next(self.gen) May 23 12:20:12.807992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 23 12:20:12.807992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self.session.rollback() May 23 12:20:12.807992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 23 12:20:12.807992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self.force_reraise() May 23 12:20:12.807992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 23 12:20:12.807992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise self.value May 23 12:20:12.807992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 23 12:20:12.807992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 23 12:20:12.807992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 23 12:20:12.807992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application session.commit() May 23 12:20:12.807992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 23 12:20:12.807992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 23 12:20:12.807992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 23 12:20:12.808918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._prepare_impl() May 23 12:20:12.808918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 23 12:20:12.808918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self.session.flush() May 23 12:20:12.808918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 23 12:20:12.808918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._flush(objects) May 23 12:20:12.808918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 23 12:20:12.808918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 23 12:20:12.808918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 23 12:20:12.808918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application compat.raise_( May 23 12:20:12.808918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 12:20:12.808918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception May 23 12:20:12.808918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 23 12:20:12.808918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application flush_context.execute() May 23 12:20:12.808918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 23 12:20:12.808918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rec.execute(self) May 23 12:20:12.808918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 23 12:20:12.808918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 23 12:20:12.809813 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 23 12:20:12.809813 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application _emit_insert_statements( May 23 12:20:12.809813 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 23 12:20:12.809813 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application result = connection._execute_20( May 23 12:20:12.809813 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 23 12:20:12.809813 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 23 12:20:12.809813 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 23 12:20:12.809813 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 23 12:20:12.809813 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 23 12:20:12.809813 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ret = self._execute_context( May 23 12:20:12.809813 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 23 12:20:12.809813 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 23 12:20:12.809813 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 23 12:20:12.809813 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 23 12:20:12.809813 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 12:20:12.809813 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception May 23 12:20:12.812095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 12:20:12.812095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 12:20:12.812095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 12:20:12.812095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 12:20:12.812095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 12:20:12.812095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application result = self._query(query) May 23 12:20:12.812095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 12:20:12.812095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application conn.query(q) May 23 12:20:12.812095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 12:20:12.812095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 12:20:12.812095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 12:20:12.812095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application result.read() May 23 12:20:12.812095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 12:20:12.812095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 12:20:12.812095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 12:20:12.812095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 12:20:12.814172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 12:20:12.814172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 12:20:12.814172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 12:20:12.814172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 12:20:12.814172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 23 12:20:12.814172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 23 12:20:12.814172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c2ee80cc85f64f0f92f8a094c00208d8', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 23 12:20:12.814172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 23 12:20:12.814172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:12.814172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:20:12.814172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:12.814172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:12.814172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:12.814172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:12.814172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:12.816098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:12.816098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:12.816098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:12.816098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:12.816098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:12.816098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:12.816098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:12.816098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 23 12:20:12.816098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 23 12:20:12.816098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:12.816098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:12.816098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:20:12.816098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:20:12.816098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:20:12.816098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:20:12.816098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 23 12:20:12.816098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 23 12:20:12.818685 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 23 12:20:12.818685 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 23 12:20:12.818685 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 23 12:20:12.818685 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 23 12:20:12.818685 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 23 12:20:12.818685 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:12.821572 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 490/973] 10.222.0.16 () {60 vars in 1131 bytes} [Mon May 23 12:20:12 2022] POST /identity/v3/users => generated 160 bytes in 202 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 23 12:20:12.827182 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef62644-46c9-41f2-bb86-43ecb787e920 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:12.835432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87818836-95a8-4b0e-8e90-75c4f2fd4ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:12.836246 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87818836-95a8-4b0e-8e90-75c4f2fd4ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:12.836664 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87818836-95a8-4b0e-8e90-75c4f2fd4ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:12.837037 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87818836-95a8-4b0e-8e90-75c4f2fd4ee7 None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:12.837873 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87818836-95a8-4b0e-8e90-75c4f2fd4ee7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:12.838291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87818836-95a8-4b0e-8e90-75c4f2fd4ee7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:12.839759 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87818836-95a8-4b0e-8e90-75c4f2fd4ee7 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:12.855756 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 484/974] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:20:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:12.860901 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87818836-95a8-4b0e-8e90-75c4f2fd4ee7 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:12.881046 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:12.948193 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef62644-46c9-41f2-bb86-43ecb787e920 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:12.955452 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:12.977068 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87818836-95a8-4b0e-8e90-75c4f2fd4ee7 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:12.987040 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff69f6e-e1dd-4a2a-a504-406477084610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:12.988134 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff69f6e-e1dd-4a2a-a504-406477084610 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:12.988865 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff69f6e-e1dd-4a2a-a504-406477084610 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:12.989544 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff69f6e-e1dd-4a2a-a504-406477084610 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:12.998105 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff69f6e-e1dd-4a2a-a504-406477084610 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:13.000076 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff69f6e-e1dd-4a2a-a504-406477084610 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:13.015004 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce3592-3c8c-433f-88fb-3b1829140527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:13.015881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce3592-3c8c-433f-88fb-3b1829140527 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:13.016278 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce3592-3c8c-433f-88fb-3b1829140527 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:13.016636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce3592-3c8c-433f-88fb-3b1829140527 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:13.016994 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-bff69f6e-e1dd-4a2a-a504-406477084610 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 12:20:13.016994 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:13.016994 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:13.016994 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:13.016994 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:13.016994 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:13.016994 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:13.016994 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:13.016994 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:13.016994 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:13.016994 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:13.016994 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:13.016994 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:20:13.016994 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:20:13.016994 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:20:13.016994 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:20:13.018109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:13.018109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:13.018109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:20:13.018109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:20:13.018109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:20:13.018109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:20:13.018109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:13.018109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:13.018109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:13.018109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:13.018109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:13.018109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:20:13.018109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:13.018109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:13.018109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:13.018109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:13.018109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:13.018109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:13.019275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:13.019275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:13.019275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:20:13.019275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:20:13.019275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:20:13.019275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:20:13.019275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:20:13.019275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:20:13.019275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 12:20:13.019275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:13.022681 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 491/975] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:20:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:13.028222 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff69f6e-e1dd-4a2a-a504-406477084610 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:13.032333 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce3592-3c8c-433f-88fb-3b1829140527 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:13.034035 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce3592-3c8c-433f-88fb-3b1829140527 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:13.049332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-b6ce3592-3c8c-433f-88fb-3b1829140527 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:20:13.049332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:13.049332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:13.049332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:13.049332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:13.049332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:13.049332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:13.049332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:13.049332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:13.049332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:13.049332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:13.049332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:13.049332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:20:13.049332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:20:13.049332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:20:13.050478 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:20:13.050478 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:13.050478 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:13.050478 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:13.050478 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:13.050478 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:13.050478 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:13.050478 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:13.050478 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:20:13.050478 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:13.050478 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:13.050478 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:13.050478 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:13.050478 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:13.050478 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:13.050478 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:13.050478 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:13.051927 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:20:13.051927 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:20:13.051927 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:20:13.051927 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:20:13.051927 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:20:13.051927 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:20:13.051927 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:20:13.051927 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:13.053609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 485/976] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:20:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:20:13.058777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce3592-3c8c-433f-88fb-3b1829140527 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:13.138073 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:13.141272 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff69f6e-e1dd-4a2a-a504-406477084610 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:13.171863 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce3592-3c8c-433f-88fb-3b1829140527 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:13.178997 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad608cfb-9d2b-4457-a2e1-653bb1d60e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:13.179904 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad608cfb-9d2b-4457-a2e1-653bb1d60e22 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:13.180309 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad608cfb-9d2b-4457-a2e1-653bb1d60e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:13.180685 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad608cfb-9d2b-4457-a2e1-653bb1d60e22 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:13.181677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad608cfb-9d2b-4457-a2e1-653bb1d60e22 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:13.182083 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad608cfb-9d2b-4457-a2e1-653bb1d60e22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:13.183701 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad608cfb-9d2b-4457-a2e1-653bb1d60e22 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:13.207800 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324ecb6-3f0a-45fd-8a35-d92909fd9d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:13.208577 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324ecb6-3f0a-45fd-8a35-d92909fd9d1f None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:13.208862 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324ecb6-3f0a-45fd-8a35-d92909fd9d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:13.209140 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324ecb6-3f0a-45fd-8a35-d92909fd9d1f None admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:13.209921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e324ecb6-3f0a-45fd-8a35-d92909fd9d1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:13.210230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e324ecb6-3f0a-45fd-8a35-d92909fd9d1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:13.211757 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e324ecb6-3f0a-45fd-8a35-d92909fd9d1f None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:13.227396 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 492/977] 10.222.0.16 () {56 vars in 1101 bytes} [Mon May 23 12:20:13 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:13.230448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 486/978] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:20:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:13.233293 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad608cfb-9d2b-4457-a2e1-653bb1d60e22 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:13.358691 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad608cfb-9d2b-4457-a2e1-653bb1d60e22 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:13.399641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f38af9-9778-4e88-a0cd-10d3e858081c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:13.400421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f38af9-9778-4e88-a0cd-10d3e858081c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:13.400702 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f38af9-9778-4e88-a0cd-10d3e858081c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:13.401026 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f38af9-9778-4e88-a0cd-10d3e858081c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:13.401883 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f38af9-9778-4e88-a0cd-10d3e858081c None admin] RBAC: Adding query filter params (role.id=3d40411f939f4ff6af2e78c756f6cc9e, scope.project.id=839585501f414e1ab02501422510e54f, user.id=733f8296ec3d4795a1140c1e1cb6eb95) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:13.402197 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f38af9-9778-4e88-a0cd-10d3e858081c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:13.403636 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f38af9-9778-4e88-a0cd-10d3e858081c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:13.410671 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 493/979] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:20:13 2022] GET /identity/v3/role_assignments?user.id=733f8296ec3d4795a1140c1e1cb6eb95&scope.project.id=839585501f414e1ab02501422510e54f&role.id=3d40411f939f4ff6af2e78c756f6cc9e => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:13.975174 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9ccda-3c6d-4456-b9ea-226c84718198 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:13.975810 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9ccda-3c6d-4456-b9ea-226c84718198 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:13.975810 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9ccda-3c6d-4456-b9ea-226c84718198 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:13.977148 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 487/980] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:13.982473 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9a0fa-d931-40e9-bf7c-a92e0a5c4204 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:13.982887 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9a0fa-d931-40e9-bf7c-a92e0a5c4204 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:13.983077 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9a0fa-d931-40e9-bf7c-a92e0a5c4204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:14.000885 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-22c9a0fa-d931-40e9-bf7c-a92e0a5c4204 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:14.002380 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-22c9a0fa-d931-40e9-bf7c-a92e0a5c4204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:14.017286 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:14.075713 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 494/981] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:20:14.099315 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e9ccda-3c6d-4456-b9ea-226c84718198 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:14.102363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:14.206685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e9ccda-3c6d-4456-b9ea-226c84718198 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:14.241623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd6d50-8a02-468c-92f0-2239d4d659db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:14.242352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd6d50-8a02-468c-92f0-2239d4d659db None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:14.242637 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd6d50-8a02-468c-92f0-2239d4d659db None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:14.242922 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd6d50-8a02-468c-92f0-2239d4d659db None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:14.243579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badd6d50-8a02-468c-92f0-2239d4d659db None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:14.244984 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badd6d50-8a02-468c-92f0-2239d4d659db None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:14.257304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-badd6d50-8a02-468c-92f0-2239d4d659db None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 12:20:14.257304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:14.257304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 12:20:14.257304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 12:20:14.257304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 12:20:14.257304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 12:20:14.257304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 12:20:14.257304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:20:14.257304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 12:20:14.257304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:14.257304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:20:14.257304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:14.257304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:14.257304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:14.257304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:14.257304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:14.257304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:14.258352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:14.258352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:14.258352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:14.258352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:14.258352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:14.258352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:14.258352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 12:20:14.258352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 12:20:14.258352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 12:20:14.258352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 12:20:14.258352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:14.258352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:14.258352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:14.258352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:14.258352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:14.258352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:14.259241 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:14.259241 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:20:14.259241 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:14.259241 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:14.259241 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:14.259241 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:14.259241 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:14.259241 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:14.259241 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:14.259241 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:14.259241 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 12:20:14.259241 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:20:14.259241 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 12:20:14.259241 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:14.259241 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 488/982] 10.222.0.16 () {56 vars in 1126 bytes} [Mon May 23 12:20:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:14.266623 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9a0fa-d931-40e9-bf7c-a92e0a5c4204 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:14.373629 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9a0fa-d931-40e9-bf7c-a92e0a5c4204 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:14.415575 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b149f804-e562-40b1-bc53-446d3e502bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:14.416275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b149f804-e562-40b1-bc53-446d3e502bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:14.416564 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b149f804-e562-40b1-bc53-446d3e502bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:14.416844 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b149f804-e562-40b1-bc53-446d3e502bf9 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:14.417553 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b149f804-e562-40b1-bc53-446d3e502bf9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:14.417869 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b149f804-e562-40b1-bc53-446d3e502bf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:14.419227 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b149f804-e562-40b1-bc53-446d3e502bf9 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:14.424288 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 495/983] 10.222.0.16 () {56 vars in 1122 bytes} [Mon May 23 12:20:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:14.430329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd6d50-8a02-468c-92f0-2239d4d659db None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:14.486573 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ef54e-38cc-4100-9749-e92e3402b3c8 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:14.486973 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ef54e-38cc-4100-9749-e92e3402b3c8 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:14.487137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ef54e-38cc-4100-9749-e92e3402b3c8 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:14.488564 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 496/984] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:14.494431 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bb0f8-46e5-40e9-afb4-6720f5f0adc3 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:14.494662 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bb0f8-46e5-40e9-afb4-6720f5f0adc3 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:14.494944 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bb0f8-46e5-40e9-afb4-6720f5f0adc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:14.501981 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:14.512249 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-e71bb0f8-46e5-40e9-afb4-6720f5f0adc3 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:14.513672 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e71bb0f8-46e5-40e9-afb4-6720f5f0adc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:14.524269 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:14.541458 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd6d50-8a02-468c-92f0-2239d4d659db None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:14.581792 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee440d2-8630-4be0-9ec4-1149081dd80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:14.582531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee440d2-8630-4be0-9ec4-1149081dd80c None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:14.583325 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee440d2-8630-4be0-9ec4-1149081dd80c None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:14.583616 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee440d2-8630-4be0-9ec4-1149081dd80c None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:14.584367 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee440d2-8630-4be0-9ec4-1149081dd80c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:14.584682 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee440d2-8630-4be0-9ec4-1149081dd80c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:14.586068 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee440d2-8630-4be0-9ec4-1149081dd80c None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:14.591439 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 489/985] 10.222.0.16 () {56 vars in 1122 bytes} [Mon May 23 12:20:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:14.598754 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 497/986] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 12:20:14.598936 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee440d2-8630-4be0-9ec4-1149081dd80c None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:14.622251 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71bb0f8-46e5-40e9-afb4-6720f5f0adc3 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:14.737398 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee440d2-8630-4be0-9ec4-1149081dd80c None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:14.755978 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71bb0f8-46e5-40e9-afb4-6720f5f0adc3 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:14.771534 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:14.778043 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36979921-f454-44b1-978b-e4a3ffc0564c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:14.778719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36979921-f454-44b1-978b-e4a3ffc0564c None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:14.779006 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36979921-f454-44b1-978b-e4a3ffc0564c None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:14.779288 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36979921-f454-44b1-978b-e4a3ffc0564c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:14.780108 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36979921-f454-44b1-978b-e4a3ffc0564c None admin] RBAC: Adding query filter params (interface=admin, service_id=ad456971d51f440ea92e0c02b48e6f92, region_id=RegionOne) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:14.780470 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36979921-f454-44b1-978b-e4a3ffc0564c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:14.782367 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36979921-f454-44b1-978b-e4a3ffc0564c None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:14.788418 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 490/987] 10.222.0.16 () {56 vars in 1249 bytes} [Mon May 23 12:20:14 2022] GET /identity/v3/endpoints?service_id=ad456971d51f440ea92e0c02b48e6f92&interface=admin®ion_id=RegionOne => generated 193 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:14.795524 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36979921-f454-44b1-978b-e4a3ffc0564c None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:14.803739 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e8fb7-1069-4f27-bbc1-af6eebfa6585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:14.804445 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e8fb7-1069-4f27-bbc1-af6eebfa6585 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:14.804737 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e8fb7-1069-4f27-bbc1-af6eebfa6585 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:14.805018 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e8fb7-1069-4f27-bbc1-af6eebfa6585 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:14.806110 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e8fb7-1069-4f27-bbc1-af6eebfa6585 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:14.807651 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e8fb7-1069-4f27-bbc1-af6eebfa6585 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:14.822443 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 498/988] 10.222.0.16 () {60 vars in 1141 bytes} [Mon May 23 12:20:14 2022] POST /identity/v3/services => generated 242 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:20:14.919838 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36979921-f454-44b1-978b-e4a3ffc0564c None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:14.983765 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d993d31-a70f-482c-baf0-ad9cd26f71be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:14.984609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d993d31-a70f-482c-baf0-ad9cd26f71be None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:14.984992 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d993d31-a70f-482c-baf0-ad9cd26f71be None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:14.985281 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d993d31-a70f-482c-baf0-ad9cd26f71be None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:14.987709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d993d31-a70f-482c-baf0-ad9cd26f71be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:14.991280 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d993d31-a70f-482c-baf0-ad9cd26f71be None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:15.007711 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 491/989] 10.222.0.16 () {56 vars in 1087 bytes} [Mon May 23 12:20:14 2022] GET /identity/v3/services => generated 1727 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:20:15.807746 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f1ad5-1ea9-4653-9805-8e176a14e9ec None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:15.808301 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f1ad5-1ea9-4653-9805-8e176a14e9ec None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:15.808416 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f1ad5-1ea9-4653-9805-8e176a14e9ec None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:15.809961 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 499/990] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:15.815115 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217c3f3-bea2-4034-9af4-3f6693cc12b7 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:15.815410 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217c3f3-bea2-4034-9af4-3f6693cc12b7 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:15.815690 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217c3f3-bea2-4034-9af4-3f6693cc12b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:15.833584 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-d217c3f3-bea2-4034-9af4-3f6693cc12b7 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:15.834766 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-d217c3f3-bea2-4034-9af4-3f6693cc12b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:15.886280 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:15.905168 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 492/991] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:15 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:20:15.927186 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f1ad5-1ea9-4653-9805-8e176a14e9ec None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:15.931083 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:16.039037 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f1ad5-1ea9-4653-9805-8e176a14e9ec None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:16.074498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19104c6-9f9c-46f2-ab3b-836467263db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:16.075209 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19104c6-9f9c-46f2-ab3b-836467263db6 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:16.075491 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19104c6-9f9c-46f2-ab3b-836467263db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:16.075770 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19104c6-9f9c-46f2-ab3b-836467263db6 None admin] PATH_INFO: `/v3/services/image` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:16.076455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19104c6-9f9c-46f2-ab3b-836467263db6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:16.077815 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19104c6-9f9c-46f2-ab3b-836467263db6 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:16.089375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-a19104c6-9f9c-46f2-ab3b-836467263db6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 23 12:20:16.089375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:16.089375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 12:20:16.089375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 12:20:16.089375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 12:20:16.089375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 12:20:16.089375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 12:20:16.089375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:20:16.089375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 12:20:16.089375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:16.089375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:20:16.089375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:16.089375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:16.089375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:16.089375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:16.089375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:16.089375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:16.090375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:16.090375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:16.090375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:16.090375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:16.090375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:16.090375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:16.090375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 12:20:16.090375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 12:20:16.090375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 12:20:16.090375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 12:20:16.090375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:16.090375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:16.090375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:16.090375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:16.090375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:16.090375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:16.091237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:16.091237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:20:16.091237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:16.091237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:16.091237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:16.091237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:16.091237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:16.091237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:16.091237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:16.091237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:16.091237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 12:20:16.091237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:20:16.091237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 12:20:16.091237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:16.095613 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 500/992] 10.222.0.16 () {56 vars in 1105 bytes} [Mon May 23 12:20:15 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:16.096449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217c3f3-bea2-4034-9af4-3f6693cc12b7 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:16.200465 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217c3f3-bea2-4034-9af4-3f6693cc12b7 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:16.235195 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e56ea-f389-432d-8ce2-91b6b636848e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:16.235870 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e56ea-f389-432d-8ce2-91b6b636848e None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:16.236156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e56ea-f389-432d-8ce2-91b6b636848e None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:16.236471 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e56ea-f389-432d-8ce2-91b6b636848e None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:16.237182 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e56ea-f389-432d-8ce2-91b6b636848e None admin] RBAC: Adding query filter params (name=image) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:16.237494 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e56ea-f389-432d-8ce2-91b6b636848e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:16.239039 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e56ea-f389-432d-8ce2-91b6b636848e None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:16.244134 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 493/993] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:20:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:16.249892 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19104c6-9f9c-46f2-ab3b-836467263db6 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:16.318817 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:16.353468 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19104c6-9f9c-46f2-ab3b-836467263db6 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:16.387815 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31785a35-a9e4-4892-9c09-0a857b4c16a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:16.388318 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31785a35-a9e4-4892-9c09-0a857b4c16a5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:16.388602 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31785a35-a9e4-4892-9c09-0a857b4c16a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:16.388873 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31785a35-a9e4-4892-9c09-0a857b4c16a5 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:16.389570 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31785a35-a9e4-4892-9c09-0a857b4c16a5 None admin] RBAC: Adding query filter params (type=image) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:16.389873 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31785a35-a9e4-4892-9c09-0a857b4c16a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:16.391231 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31785a35-a9e4-4892-9c09-0a857b4c16a5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:16.398800 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 501/994] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:20:16 2022] GET /identity/v3/services?type=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:17.136269 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426bda1-15e9-4a22-9837-45c676df5f79 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:17.136816 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426bda1-15e9-4a22-9837-45c676df5f79 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:17.136816 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426bda1-15e9-4a22-9837-45c676df5f79 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:17.138174 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 494/995] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:17.143290 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a3a35-ce31-4b1f-b943-c4261acaab97 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:17.143692 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a3a35-ce31-4b1f-b943-c4261acaab97 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:17.143867 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a3a35-ce31-4b1f-b943-c4261acaab97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:17.161111 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-9a5a3a35-ce31-4b1f-b943-c4261acaab97 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:17.162282 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-9a5a3a35-ce31-4b1f-b943-c4261acaab97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:17.230122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 502/996] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:20:17.252673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426bda1-15e9-4a22-9837-45c676df5f79 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:17.286336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73937156-8b42-43c5-a691-12c98606be23 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:17.286568 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73937156-8b42-43c5-a691-12c98606be23 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:17.286850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73937156-8b42-43c5-a691-12c98606be23 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:17.288249 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 503/997] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:20:17.293267 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2628775-300a-49db-8a6b-580ebc60ff02 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:17.293556 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2628775-300a-49db-8a6b-580ebc60ff02 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:17.293837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2628775-300a-49db-8a6b-580ebc60ff02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:17.311153 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:17.314625 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-c2628775-300a-49db-8a6b-580ebc60ff02 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:17.316405 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c2628775-300a-49db-8a6b-580ebc60ff02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:17.365707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426bda1-15e9-4a22-9837-45c676df5f79 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:17.390839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:17.402557 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fde842-b640-48b3-accb-72827bf219a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:17.403353 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fde842-b640-48b3-accb-72827bf219a1 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:17.403633 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fde842-b640-48b3-accb-72827bf219a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:17.403912 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fde842-b640-48b3-accb-72827bf219a1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:17.404604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fde842-b640-48b3-accb-72827bf219a1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:17.405144 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 504/998] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 12:20:17.405981 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fde842-b640-48b3-accb-72827bf219a1 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:17.417947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-f6fde842-b640-48b3-accb-72827bf219a1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 23 12:20:17.417947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:17.417947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 12:20:17.417947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 12:20:17.417947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 12:20:17.417947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 12:20:17.417947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 12:20:17.417947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:20:17.417947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 12:20:17.417947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:17.417947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:20:17.417947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:17.417947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:17.417947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:17.417947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:17.417947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:17.417947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:17.418962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:17.418962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:17.418962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:17.418962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:17.418962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:17.418962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:17.418962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 12:20:17.418962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 12:20:17.418962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 12:20:17.418962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 12:20:17.418962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:17.418962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:17.418962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:17.418962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:17.418962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:17.418962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:17.420429 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:17.420429 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:20:17.420429 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:17.420429 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:17.420429 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:17.420429 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:17.420429 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:17.420429 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:17.420429 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:17.420429 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:17.420429 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 12:20:17.420429 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:20:17.420429 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 12:20:17.420429 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:17.420429 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 495/999] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:20:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:17.424948 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2628775-300a-49db-8a6b-580ebc60ff02 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:17.429336 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fde842-b640-48b3-accb-72827bf219a1 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:17.533104 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2628775-300a-49db-8a6b-580ebc60ff02 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:17.538743 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fde842-b640-48b3-accb-72827bf219a1 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:17.553164 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:17.575029 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977dd5dc-1669-4c5e-bc12-07a369568fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:17.575712 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977dd5dc-1669-4c5e-bc12-07a369568fbc None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:17.576021 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977dd5dc-1669-4c5e-bc12-07a369568fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:17.576339 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977dd5dc-1669-4c5e-bc12-07a369568fbc None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:17.577052 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977dd5dc-1669-4c5e-bc12-07a369568fbc None admin] RBAC: Adding query filter params (name=compute) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:17.577369 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977dd5dc-1669-4c5e-bc12-07a369568fbc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:17.577585 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335883f-0d90-4dd6-b0d3-b92014619793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:17.578392 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335883f-0d90-4dd6-b0d3-b92014619793 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:17.578678 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335883f-0d90-4dd6-b0d3-b92014619793 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:17.578772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977dd5dc-1669-4c5e-bc12-07a369568fbc None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:17.578955 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335883f-0d90-4dd6-b0d3-b92014619793 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:17.579611 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c335883f-0d90-4dd6-b0d3-b92014619793 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:17.581074 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c335883f-0d90-4dd6-b0d3-b92014619793 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:17.588532 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 505/1000] 10.222.0.16 () {56 vars in 1112 bytes} [Mon May 23 12:20:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:17.589521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977dd5dc-1669-4c5e-bc12-07a369568fbc None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:17.593546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-c335883f-0d90-4dd6-b0d3-b92014619793 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 12:20:17.593546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:17.593546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 12:20:17.593546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 12:20:17.593546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 12:20:17.593546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 12:20:17.593546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 12:20:17.593546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:20:17.593546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 12:20:17.593546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:17.593546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:20:17.593546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:17.593546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:17.593546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:17.593546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:17.593546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:17.593546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:17.594569 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:17.594569 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:17.594569 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:17.594569 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:17.594569 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:17.594569 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:17.594569 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 12:20:17.594569 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 12:20:17.594569 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 12:20:17.594569 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 12:20:17.594569 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:17.594569 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:17.594569 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:17.594569 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:17.594569 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:17.594569 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:17.595813 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:17.595813 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:20:17.595813 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:17.595813 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:17.595813 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:17.595813 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:17.595813 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:17.595813 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:17.595813 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:17.595813 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:17.595813 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 12:20:17.595813 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:20:17.595813 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 12:20:17.595813 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:17.595813 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 496/1001] 10.222.0.16 () {56 vars in 1126 bytes} [Mon May 23 12:20:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:20:17.601016 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335883f-0d90-4dd6-b0d3-b92014619793 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:17.656496 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:17.704581 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977dd5dc-1669-4c5e-bc12-07a369568fbc None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:17.706019 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335883f-0d90-4dd6-b0d3-b92014619793 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:17.742028 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe4c55-c539-48c9-8bf0-1b7565553812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:17.742579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49b0d2-1a78-46cf-9622-acacaa9ae5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:17.742721 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe4c55-c539-48c9-8bf0-1b7565553812 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:17.742964 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe4c55-c539-48c9-8bf0-1b7565553812 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:17.743239 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49b0d2-1a78-46cf-9622-acacaa9ae5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:17.743345 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe4c55-c539-48c9-8bf0-1b7565553812 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:17.743526 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49b0d2-1a78-46cf-9622-acacaa9ae5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:17.743802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49b0d2-1a78-46cf-9622-acacaa9ae5f2 None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:17.743958 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe4c55-c539-48c9-8bf0-1b7565553812 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:17.744267 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe4c55-c539-48c9-8bf0-1b7565553812 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:17.744537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49b0d2-1a78-46cf-9622-acacaa9ae5f2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:17.744845 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49b0d2-1a78-46cf-9622-acacaa9ae5f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:17.745638 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe4c55-c539-48c9-8bf0-1b7565553812 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:17.746197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49b0d2-1a78-46cf-9622-acacaa9ae5f2 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:17.751089 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 506/1002] 10.222.0.16 () {56 vars in 1112 bytes} [Mon May 23 12:20:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:17.751335 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 497/1003] 10.222.0.16 () {56 vars in 1122 bytes} [Mon May 23 12:20:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:17.757163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe4c55-c539-48c9-8bf0-1b7565553812 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:17.757595 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49b0d2-1a78-46cf-9622-acacaa9ae5f2 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:17.805248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:17.864736 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe4c55-c539-48c9-8bf0-1b7565553812 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:17.871016 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49b0d2-1a78-46cf-9622-acacaa9ae5f2 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:17.885401 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:17.902108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0508f2-193e-4c60-a394-38685b63a2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:17.902783 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0508f2-193e-4c60-a394-38685b63a2dc None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:17.903067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0508f2-193e-4c60-a394-38685b63a2dc None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:17.903353 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0508f2-193e-4c60-a394-38685b63a2dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:17.904122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0508f2-193e-4c60-a394-38685b63a2dc None admin] RBAC: Adding query filter params (interface=public, service_id=cd5924ed862448a5a6e662faca52695a, region_id=RegionOne) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:17.904465 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0508f2-193e-4c60-a394-38685b63a2dc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:17.905832 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0508f2-193e-4c60-a394-38685b63a2dc None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:17.909350 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede289d-5a1e-4949-8afc-109674da56f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:17.910381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede289d-5a1e-4949-8afc-109674da56f4 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:17.911422 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede289d-5a1e-4949-8afc-109674da56f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:17.911896 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede289d-5a1e-4949-8afc-109674da56f4 None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:17.912013 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 507/1004] 10.222.0.16 () {56 vars in 1251 bytes} [Mon May 23 12:20:17 2022] GET /identity/v3/endpoints?service_id=cd5924ed862448a5a6e662faca52695a&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:17.912978 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dede289d-5a1e-4949-8afc-109674da56f4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:17.913301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dede289d-5a1e-4949-8afc-109674da56f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:17.914678 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dede289d-5a1e-4949-8afc-109674da56f4 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:17.917198 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0508f2-193e-4c60-a394-38685b63a2dc None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:17.921243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 498/1005] 10.222.0.16 () {56 vars in 1122 bytes} [Mon May 23 12:20:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:17.926629 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede289d-5a1e-4949-8afc-109674da56f4 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:18.033517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede289d-5a1e-4949-8afc-109674da56f4 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:18.036845 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:18.041069 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0508f2-193e-4c60-a394-38685b63a2dc None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:18.076159 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a2d7fa-5f78-491f-8e60-91edf3e142b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:18.077166 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a2d7fa-5f78-491f-8e60-91edf3e142b5 None admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:18.077516 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a2d7fa-5f78-491f-8e60-91edf3e142b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:18.077740 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a2d7fa-5f78-491f-8e60-91edf3e142b5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:18.078490 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a2d7fa-5f78-491f-8e60-91edf3e142b5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:18.079871 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a2d7fa-5f78-491f-8e60-91edf3e142b5 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:18.080956 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8747d2-6215-4904-b34e-13b3ef77f7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:18.081624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8747d2-6215-4904-b34e-13b3ef77f7e7 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:18.081905 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8747d2-6215-4904-b34e-13b3ef77f7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:18.082177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8747d2-6215-4904-b34e-13b3ef77f7e7 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:18.083277 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8747d2-6215-4904-b34e-13b3ef77f7e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:18.085712 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8747d2-6215-4904-b34e-13b3ef77f7e7 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:18.103697 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 508/1006] 10.222.0.16 () {56 vars in 1087 bytes} [Mon May 23 12:20:17 2022] GET /identity/v3/services => generated 1727 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:20:18.103923 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 499/1007] 10.222.0.16 () {60 vars in 1144 bytes} [Mon May 23 12:20:17 2022] POST /identity/v3/endpoints => generated 325 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:20:18.615828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684426c8-8e6c-4d2e-978b-522665723211 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:18.616378 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684426c8-8e6c-4d2e-978b-522665723211 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:18.616458 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684426c8-8e6c-4d2e-978b-522665723211 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:18.617800 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 509/1008] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:18.622991 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b6443-d188-46c4-bff3-7659231f8528 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:18.623458 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b6443-d188-46c4-bff3-7659231f8528 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:18.623539 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b6443-d188-46c4-bff3-7659231f8528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:18.641694 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-344b6443-d188-46c4-bff3-7659231f8528 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:18.642878 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-344b6443-d188-46c4-bff3-7659231f8528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:18.728331 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 500/1009] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:18 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:20:18.749923 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684426c8-8e6c-4d2e-978b-522665723211 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:18.801739 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:18.854988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684426c8-8e6c-4d2e-978b-522665723211 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:18.889302 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af57a11-223a-4929-bca2-f29462d7fac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:18.889880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af57a11-223a-4929-bca2-f29462d7fac5 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:18.890160 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af57a11-223a-4929-bca2-f29462d7fac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:18.890433 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af57a11-223a-4929-bca2-f29462d7fac5 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:18.891157 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af57a11-223a-4929-bca2-f29462d7fac5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:18.892888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af57a11-223a-4929-bca2-f29462d7fac5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:18.916210 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 510/1010] 10.222.0.16 () {60 vars in 1141 bytes} [Mon May 23 12:20:18 2022] POST /identity/v3/services => generated 242 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:20:20.358007 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f5150-082e-4d86-880d-869db9ce520f None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:20.358512 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f5150-082e-4d86-880d-869db9ce520f None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:20.358623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f5150-082e-4d86-880d-869db9ce520f None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:20.359921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 501/1011] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:20.366269 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a50164-4841-49f2-804f-0eb7be8c528a None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:20.366685 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a50164-4841-49f2-804f-0eb7be8c528a None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:20.366825 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a50164-4841-49f2-804f-0eb7be8c528a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:20.383887 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-e6a50164-4841-49f2-804f-0eb7be8c528a None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:20.385112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e6a50164-4841-49f2-804f-0eb7be8c528a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:20.405329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1521448-0421-4f04-aa4e-217d4b194339 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:20.405614 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1521448-0421-4f04-aa4e-217d4b194339 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:20.406069 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1521448-0421-4f04-aa4e-217d4b194339 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:20.407702 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 502/1012] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:20.413232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05f4cb-1d0c-4de8-94ce-e2ba326aeb89 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:20.413524 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05f4cb-1d0c-4de8-94ce-e2ba326aeb89 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:20.413967 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05f4cb-1d0c-4de8-94ce-e2ba326aeb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:20.432989 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-fc05f4cb-1d0c-4de8-94ce-e2ba326aeb89 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:20.434796 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-fc05f4cb-1d0c-4de8-94ce-e2ba326aeb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:20.466463 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 511/1013] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:20 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 12:20:20.494082 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a50164-4841-49f2-804f-0eb7be8c528a None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:20.516235 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 503/1014] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:20 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 12:20:20.530309 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:20.536774 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05f4cb-1d0c-4de8-94ce-e2ba326aeb89 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:20.559075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:20.603347 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a50164-4841-49f2-804f-0eb7be8c528a None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:20.641283 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e671c-8ed8-4a57-b352-74afb08cdfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:20.642452 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e671c-8ed8-4a57-b352-74afb08cdfab None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:20.642888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e671c-8ed8-4a57-b352-74afb08cdfab None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:20.643302 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e671c-8ed8-4a57-b352-74afb08cdfab None admin] PATH_INFO: `/v3/services/compute` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:20.644453 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e671c-8ed8-4a57-b352-74afb08cdfab None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:20.647460 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e671c-8ed8-4a57-b352-74afb08cdfab None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:20.653515 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc05f4cb-1d0c-4de8-94ce-e2ba326aeb89 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:20.662601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-c58e671c-8ed8-4a57-b352-74afb08cdfab None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 23 12:20:20.662601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:20.662601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 12:20:20.662601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 12:20:20.662601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 12:20:20.662601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 12:20:20.662601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 12:20:20.662601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:20:20.662601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 12:20:20.662601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:20.662601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:20:20.662601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:20.662601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:20.662601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:20.662601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:20.662601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:20.662601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:20.663684 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:20.663684 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:20.663684 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:20.663684 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:20.663684 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:20.663684 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:20.663684 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 12:20:20.663684 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 12:20:20.663684 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 12:20:20.663684 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 12:20:20.663684 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:20.663684 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:20.663684 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:20.663684 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:20.663684 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:20.663684 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:20.664616 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:20.664616 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:20:20.664616 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:20.664616 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:20.664616 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:20.664616 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:20.664616 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:20.664616 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:20.664616 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:20.664616 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:20.664616 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 12:20:20.664616 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:20:20.664616 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 12:20:20.664616 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:20.664616 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 512/1015] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:20:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:20:20.670249 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e671c-8ed8-4a57-b352-74afb08cdfab None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:20.698893 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a797c911-cacf-4d64-b9b3-7aae442afd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:20.699691 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797c911-cacf-4d64-b9b3-7aae442afd4b None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:20.699976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797c911-cacf-4d64-b9b3-7aae442afd4b None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:20.700291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797c911-cacf-4d64-b9b3-7aae442afd4b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:20.701550 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a797c911-cacf-4d64-b9b3-7aae442afd4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:20.702915 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a797c911-cacf-4d64-b9b3-7aae442afd4b None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:20.704898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 504/1016] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:20:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:20.710771 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a797c911-cacf-4d64-b9b3-7aae442afd4b None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:20.790416 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:20.797254 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e671c-8ed8-4a57-b352-74afb08cdfab None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:20.808137 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:20.822012 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a797c911-cacf-4d64-b9b3-7aae442afd4b None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:20.837175 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6396649-e292-483e-ac98-466e4e85570b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:20.838153 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6396649-e292-483e-ac98-466e4e85570b None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:20.838153 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6396649-e292-483e-ac98-466e4e85570b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:20.838384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6396649-e292-483e-ac98-466e4e85570b None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:20.839132 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6396649-e292-483e-ac98-466e4e85570b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:20.839472 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6396649-e292-483e-ac98-466e4e85570b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:20.840897 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6396649-e292-483e-ac98-466e4e85570b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:20.851886 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 513/1017] 10.222.0.16 () {56 vars in 1112 bytes} [Mon May 23 12:20:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:20.853553 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6396649-e292-483e-ac98-466e4e85570b None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:20.860859 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf11eab9-b34b-4dbf-819a-2b336bcf737a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:20.861525 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11eab9-b34b-4dbf-819a-2b336bcf737a None admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:20.861828 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11eab9-b34b-4dbf-819a-2b336bcf737a None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:20.862109 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf11eab9-b34b-4dbf-819a-2b336bcf737a None admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:20.862984 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf11eab9-b34b-4dbf-819a-2b336bcf737a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:20.865163 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf11eab9-b34b-4dbf-819a-2b336bcf737a None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:20.906996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 505/1018] 10.222.0.16 () {60 vars in 1141 bytes} [Mon May 23 12:20:20 2022] POST /identity/v3/projects => generated 311 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:20:20.976951 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6396649-e292-483e-ac98-466e4e85570b None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:21.015045 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89d785-2a6d-4b40-85af-63005241cd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:21.015741 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89d785-2a6d-4b40-85af-63005241cd1e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:21.016016 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89d785-2a6d-4b40-85af-63005241cd1e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:21.016327 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89d785-2a6d-4b40-85af-63005241cd1e None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:21.017061 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89d785-2a6d-4b40-85af-63005241cd1e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:21.017380 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89d785-2a6d-4b40-85af-63005241cd1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:21.020291 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89d785-2a6d-4b40-85af-63005241cd1e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:21.029517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 514/1019] 10.222.0.16 () {56 vars in 1112 bytes} [Mon May 23 12:20:20 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:21.035389 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf11eab9-b34b-4dbf-819a-2b336bcf737a None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:21.149093 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf11eab9-b34b-4dbf-819a-2b336bcf737a None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:21.188766 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d299d-0c31-4408-b10e-81655f6de710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:21.189374 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d299d-0c31-4408-b10e-81655f6de710 None admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:21.189662 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d299d-0c31-4408-b10e-81655f6de710 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:21.189942 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d299d-0c31-4408-b10e-81655f6de710 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:21.190744 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864d299d-0c31-4408-b10e-81655f6de710 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:21.192974 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864d299d-0c31-4408-b10e-81655f6de710 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:21.202944 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:21.220214 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 506/1020] 10.222.0.16 () {60 vars in 1144 bytes} [Mon May 23 12:20:21 2022] POST /identity/v3/endpoints => generated 334 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:20:21.234618 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819d014-cd0e-4f57-b68a-67494c1a4c5e None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:21.234913 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819d014-cd0e-4f57-b68a-67494c1a4c5e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:21.235204 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819d014-cd0e-4f57-b68a-67494c1a4c5e None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:21.236714 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 515/1021] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:20:21.245336 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6dd94-b4a2-4189-aae5-ea9e2be7ca9f None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:21.245690 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6dd94-b4a2-4189-aae5-ea9e2be7ca9f None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:21.246038 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6dd94-b4a2-4189-aae5-ea9e2be7ca9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:21.265296 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-65e6dd94-b4a2-4189-aae5-ea9e2be7ca9f None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:21.266532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-65e6dd94-b4a2-4189-aae5-ea9e2be7ca9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:21.352339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 507/1022] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:21 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:20:21.374836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819d014-cd0e-4f57-b68a-67494c1a4c5e None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:21.408063 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:21.497025 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819d014-cd0e-4f57-b68a-67494c1a4c5e None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:21.533426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642eeaa-e6e6-4920-ab3c-6837df66f1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:21.534219 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642eeaa-e6e6-4920-ab3c-6837df66f1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:21.534502 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642eeaa-e6e6-4920-ab3c-6837df66f1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:21.534780 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642eeaa-e6e6-4920-ab3c-6837df66f1f6 None admin] PATH_INFO: `/v3/services/image` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:21.535573 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642eeaa-e6e6-4920-ab3c-6837df66f1f6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:21.537148 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642eeaa-e6e6-4920-ab3c-6837df66f1f6 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:21.549135 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-d642eeaa-e6e6-4920-ab3c-6837df66f1f6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 23 12:20:21.549135 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:21.549135 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 12:20:21.549135 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 12:20:21.549135 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 12:20:21.549135 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 12:20:21.549135 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 12:20:21.549135 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:20:21.549135 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 12:20:21.549135 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:21.549135 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:20:21.549135 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:21.549135 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:21.549135 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:21.549135 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:21.549135 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:21.549135 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:21.550398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:21.550398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:21.550398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:21.550398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:21.550398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:21.550398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:21.550398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 12:20:21.550398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 12:20:21.550398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 12:20:21.550398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 12:20:21.550398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:21.550398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:21.550398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:21.550398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:21.550398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:21.550398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:21.551695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:21.551695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:20:21.551695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:21.551695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:21.551695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:21.551695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:21.551695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:21.551695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:21.551695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:21.551695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:21.551695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 12:20:21.551695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:20:21.551695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 12:20:21.551695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:21.551695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 516/1023] 10.222.0.16 () {56 vars in 1105 bytes} [Mon May 23 12:20:21 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:21.556768 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e6dd94-b4a2-4189-aae5-ea9e2be7ca9f None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:21.661272 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e6dd94-b4a2-4189-aae5-ea9e2be7ca9f None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:21.685811 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:21.702819 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4dc11-3851-4d7a-b41c-b14049c5123c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:21.703489 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4dc11-3851-4d7a-b41c-b14049c5123c None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:21.703775 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4dc11-3851-4d7a-b41c-b14049c5123c None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:21.704055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd4dc11-3851-4d7a-b41c-b14049c5123c None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:21.704809 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd4dc11-3851-4d7a-b41c-b14049c5123c None admin] RBAC: Adding query filter params (name=image) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:21.705120 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd4dc11-3851-4d7a-b41c-b14049c5123c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:21.706502 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd4dc11-3851-4d7a-b41c-b14049c5123c None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:21.717016 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 508/1024] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:20:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:21.718638 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642eeaa-e6e6-4920-ab3c-6837df66f1f6 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:21.815103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:21.827343 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642eeaa-e6e6-4920-ab3c-6837df66f1f6 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:21.861863 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fb355-48ad-4304-bcae-bb9f8e3dac6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:21.862522 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fb355-48ad-4304-bcae-bb9f8e3dac6f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:21.864454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fb355-48ad-4304-bcae-bb9f8e3dac6f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:21.864454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fb355-48ad-4304-bcae-bb9f8e3dac6f None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:21.864454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7fb355-48ad-4304-bcae-bb9f8e3dac6f None admin] RBAC: Adding query filter params (type=image) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:21.864454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7fb355-48ad-4304-bcae-bb9f8e3dac6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:21.865676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7fb355-48ad-4304-bcae-bb9f8e3dac6f None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:21.870946 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 517/1025] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:20:21 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:21.876937 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4dc11-3851-4d7a-b41c-b14049c5123c None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:21.980843 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd4dc11-3851-4d7a-b41c-b14049c5123c None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:22.014370 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6446af20-737c-4930-9372-1a0c069d7110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:22.015026 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446af20-737c-4930-9372-1a0c069d7110 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:22.015313 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446af20-737c-4930-9372-1a0c069d7110 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:22.015590 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6446af20-737c-4930-9372-1a0c069d7110 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:22.016401 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446af20-737c-4930-9372-1a0c069d7110 None admin] RBAC: Adding query filter params (interface=public, service_id=8d00a39e4c0443a6abc9d500c635086f, region_id=RegionOne) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:22.016717 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446af20-737c-4930-9372-1a0c069d7110 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:22.021548 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6446af20-737c-4930-9372-1a0c069d7110 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:22.032700 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 509/1026] 10.222.0.16 () {56 vars in 1251 bytes} [Mon May 23 12:20:21 2022] GET /identity/v3/endpoints?service_id=8d00a39e4c0443a6abc9d500c635086f&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:22.038077 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fb355-48ad-4304-bcae-bb9f8e3dac6f None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:22.143209 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fb355-48ad-4304-bcae-bb9f8e3dac6f None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:22.178164 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08993705-2a73-4a33-a8f1-536b53932ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:22.178852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08993705-2a73-4a33-a8f1-536b53932ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:22.179149 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08993705-2a73-4a33-a8f1-536b53932ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:22.179421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08993705-2a73-4a33-a8f1-536b53932ad0 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:22.180129 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08993705-2a73-4a33-a8f1-536b53932ad0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:22.181625 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08993705-2a73-4a33-a8f1-536b53932ad0 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:22.196353 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 518/1027] 10.222.0.16 () {56 vars in 1087 bytes} [Mon May 23 12:20:22 2022] GET /identity/v3/services => generated 1957 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:20:23.426938 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6235a-9b7f-48d9-8439-9f9783dc2ac0 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:23.427512 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6235a-9b7f-48d9-8439-9f9783dc2ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:23.427512 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6235a-9b7f-48d9-8439-9f9783dc2ac0 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:23.428924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 510/1028] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:23.434971 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffdc79a-7ab5-4034-8848-d0d20dff68e4 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:23.435501 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffdc79a-7ab5-4034-8848-d0d20dff68e4 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:23.435579 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffdc79a-7ab5-4034-8848-d0d20dff68e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:23.453227 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-cffdc79a-7ab5-4034-8848-d0d20dff68e4 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:23.454403 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-cffdc79a-7ab5-4034-8848-d0d20dff68e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:23.468358 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:23.524799 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 519/1029] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:20:23.549271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6235a-9b7f-48d9-8439-9f9783dc2ac0 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:23.601641 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:23.602678 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a44ec-3268-4403-b233-d7e5f28de752 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:23.603036 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a44ec-3268-4403-b233-d7e5f28de752 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:23.603318 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a44ec-3268-4403-b233-d7e5f28de752 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:23.604948 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 520/1030] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:20:23.609733 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8a7d9-4cc2-401a-aef2-84a9f0810ff0 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:23.610030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8a7d9-4cc2-401a-aef2-84a9f0810ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:23.610314 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8a7d9-4cc2-401a-aef2-84a9f0810ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:23.627942 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-11d8a7d9-4cc2-401a-aef2-84a9f0810ff0 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:23.629243 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-11d8a7d9-4cc2-401a-aef2-84a9f0810ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:23.653655 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6235a-9b7f-48d9-8439-9f9783dc2ac0 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:23.689611 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bdcc0e-f812-43e3-8feb-72fa179a4090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:23.690380 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bdcc0e-f812-43e3-8feb-72fa179a4090 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:23.690710 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bdcc0e-f812-43e3-8feb-72fa179a4090 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:23.690946 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bdcc0e-f812-43e3-8feb-72fa179a4090 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:23.692515 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bdcc0e-f812-43e3-8feb-72fa179a4090 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:23.694018 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bdcc0e-f812-43e3-8feb-72fa179a4090 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:23.695733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 511/1031] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:20:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:23.701070 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bdcc0e-f812-43e3-8feb-72fa179a4090 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:23.702692 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 521/1032] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 12:20:23.727306 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8a7d9-4cc2-401a-aef2-84a9f0810ff0 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:23.807280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:23.813865 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bdcc0e-f812-43e3-8feb-72fa179a4090 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:23.830794 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8a7d9-4cc2-401a-aef2-84a9f0810ff0 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:23.833067 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:23.851653 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74bba0-56ca-42e2-85f7-856bd8c3aaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:23.852359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74bba0-56ca-42e2-85f7-856bd8c3aaa4 None admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:23.852647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74bba0-56ca-42e2-85f7-856bd8c3aaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:23.852927 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f74bba0-56ca-42e2-85f7-856bd8c3aaa4 None admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:23.853667 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f74bba0-56ca-42e2-85f7-856bd8c3aaa4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:23.855124 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f74bba0-56ca-42e2-85f7-856bd8c3aaa4 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:23.872081 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a1539-18cf-4c41-8540-018497cb5e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:23.872959 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a1539-18cf-4c41-8540-018497cb5e9f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:23.873247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a1539-18cf-4c41-8540-018497cb5e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:23.873530 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a1539-18cf-4c41-8540-018497cb5e9f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:23.884262 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 512/1033] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 12:20:23 2022] POST /identity/v3/users => generated 285 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:20:23.885206 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86a1539-18cf-4c41-8540-018497cb5e9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:20:23.893710 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-d86a1539-18cf-4c41-8540-018497cb5e9f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 12:20:23.893710 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:23.893710 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:23.893710 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:23.893710 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:23.893710 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:23.893710 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:23.893710 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:23.893710 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:23.893710 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:23.893710 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:23.893710 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:23.893710 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:20:23.893710 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:20:23.893710 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:20:23.893710 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:20:23.895316 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:20:23.895316 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:20:23.895316 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:20:23.895316 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:20:23.895316 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:23.895316 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:23.895316 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:23.895316 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:23.895316 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:23.895316 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:23.895316 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:23.895316 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:20:23.895316 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:23.895316 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:23.895316 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:23.895316 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:23.896306 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:23.896306 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:23.896306 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:23.896306 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:23.896306 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:20:23.896306 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:20:23.896306 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:20:23.896306 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:20:23.896306 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:20:23.896306 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:20:23.896306 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 12:20:23.896306 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:23.896306 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 522/1034] 10.222.0.16 () {56 vars in 1120 bytes} [Mon May 23 12:20:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:20:23.901824 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74bba0-56ca-42e2-85f7-856bd8c3aaa4 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:24.015704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f74bba0-56ca-42e2-85f7-856bd8c3aaa4 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:24.056577 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075beccd-a699-4324-912c-cc974803bc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:24.057262 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075beccd-a699-4324-912c-cc974803bc22 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:24.057546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075beccd-a699-4324-912c-cc974803bc22 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:24.057832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075beccd-a699-4324-912c-cc974803bc22 None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:24.060241 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075beccd-a699-4324-912c-cc974803bc22 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:24.060241 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075beccd-a699-4324-912c-cc974803bc22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:24.060429 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075beccd-a699-4324-912c-cc974803bc22 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:24.071299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 513/1035] 10.222.0.16 () {56 vars in 1115 bytes} [Mon May 23 12:20:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:24.077062 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a1539-18cf-4c41-8540-018497cb5e9f None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:24.186757 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a1539-18cf-4c41-8540-018497cb5e9f None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:24.220782 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba500a4-380b-4343-84a2-74c6f9fbe7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:24.221580 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba500a4-380b-4343-84a2-74c6f9fbe7ac None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:24.221865 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba500a4-380b-4343-84a2-74c6f9fbe7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:24.222143 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba500a4-380b-4343-84a2-74c6f9fbe7ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:24.229833 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:24.235685 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba500a4-380b-4343-84a2-74c6f9fbe7ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:24.237202 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba500a4-380b-4343-84a2-74c6f9fbe7ac None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:24.256509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-aba500a4-380b-4343-84a2-74c6f9fbe7ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:20:24.256509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:24.256509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:24.256509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:24.256509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:24.256509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:24.256509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:24.256509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:24.256509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:24.256509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:24.256509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:24.256509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:24.256509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:20:24.256509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:20:24.256509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:20:24.257498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:20:24.257498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:24.257498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:24.257498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:24.257498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:24.257498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:24.257498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:24.257498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:24.257498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:20:24.257498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:24.257498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:24.257498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:24.257498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:24.257498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:24.257498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:24.257498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:24.257498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:24.258527 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:20:24.258527 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:20:24.258527 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:20:24.258527 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:24.258527 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 523/1036] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:20:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:20:24.264919 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075beccd-a699-4324-912c-cc974803bc22 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:24.306387 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:24.376658 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075beccd-a699-4324-912c-cc974803bc22 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:24.410730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2617d59d-160b-4b33-a692-810490ecced8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:24.411426 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617d59d-160b-4b33-a692-810490ecced8 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:24.411873 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617d59d-160b-4b33-a692-810490ecced8 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:24.412156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617d59d-160b-4b33-a692-810490ecced8 None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:24.412903 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2617d59d-160b-4b33-a692-810490ecced8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:24.413213 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2617d59d-160b-4b33-a692-810490ecced8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:24.414587 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2617d59d-160b-4b33-a692-810490ecced8 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:24.426820 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 514/1037] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:24.432289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba500a4-380b-4343-84a2-74c6f9fbe7ac None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:24.453265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a340c6-edf6-43ef-8ff3-934912ca6f6f None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:24.453801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a340c6-edf6-43ef-8ff3-934912ca6f6f None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:24.453801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a340c6-edf6-43ef-8ff3-934912ca6f6f None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:24.455287 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 515/1038] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:20:24.460589 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014bf914-06f8-43e0-b8ad-59a131530862 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:24.460895 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014bf914-06f8-43e0-b8ad-59a131530862 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:24.461178 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014bf914-06f8-43e0-b8ad-59a131530862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:24.478776 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-014bf914-06f8-43e0-b8ad-59a131530862 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:24.480007 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-014bf914-06f8-43e0-b8ad-59a131530862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:24.544735 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba500a4-380b-4343-84a2-74c6f9fbe7ac None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:24.552927 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 516/1039] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:24 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 12:20:24.576317 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014bf914-06f8-43e0-b8ad-59a131530862 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:24.583683 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0fb5b2-fffd-4126-b125-5c784a9400b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:24.585328 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fb5b2-fffd-4126-b125-5c784a9400b6 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:24.585885 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fb5b2-fffd-4126-b125-5c784a9400b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:24.586373 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fb5b2-fffd-4126-b125-5c784a9400b6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:24.595723 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0fb5b2-fffd-4126-b125-5c784a9400b6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:24.597177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0fb5b2-fffd-4126-b125-5c784a9400b6 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:24.604962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:24.612515 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-ff0fb5b2-fffd-4126-b125-5c784a9400b6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 12:20:24.612515 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:24.612515 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:24.612515 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:24.612515 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:24.612515 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:24.612515 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:24.612515 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:24.612515 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:24.612515 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:24.612515 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:24.612515 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:24.612515 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:20:24.612515 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:20:24.612515 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:20:24.612515 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:20:24.613555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:24.613555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:24.613555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:20:24.613555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:20:24.613555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:20:24.613555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:20:24.613555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:24.613555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:24.613555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:24.613555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:24.613555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:24.613555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:20:24.613555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:24.613555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:24.613555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:24.613555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:24.613555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:24.613555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:24.614544 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:24.614544 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:24.614544 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:20:24.614544 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:20:24.614544 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:20:24.614544 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:20:24.614544 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:20:24.614544 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:20:24.614544 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 12:20:24.614544 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:24.614544 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 524/1040] 10.222.0.16 () {56 vars in 1093 bytes} [Mon May 23 12:20:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:24.619565 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0fb5b2-fffd-4126-b125-5c784a9400b6 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:24.681510 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:24.700397 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014bf914-06f8-43e0-b8ad-59a131530862 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:24.737205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0467e5-133b-40da-9cd4-44dd4c7a62d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:24.738269 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0467e5-133b-40da-9cd4-44dd4c7a62d4 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:24.739421 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0467e5-133b-40da-9cd4-44dd4c7a62d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:24.739861 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0467e5-133b-40da-9cd4-44dd4c7a62d4 None admin] PATH_INFO: `/v3/services/image` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:24.740554 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0467e5-133b-40da-9cd4-44dd4c7a62d4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:24.741935 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0467e5-133b-40da-9cd4-44dd4c7a62d4 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:24.750032 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0fb5b2-fffd-4126-b125-5c784a9400b6 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:24.753685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-ee0467e5-133b-40da-9cd4-44dd4c7a62d4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 23 12:20:24.753685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:24.753685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 12:20:24.753685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 12:20:24.753685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 12:20:24.753685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 12:20:24.753685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 12:20:24.753685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:20:24.753685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 12:20:24.753685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:24.753685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:20:24.753685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:24.753685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:24.753685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:24.753685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:24.753685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:24.753685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:24.754963 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:24.754963 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:24.754963 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:24.754963 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:24.754963 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:24.754963 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:24.754963 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 12:20:24.754963 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 12:20:24.754963 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 12:20:24.754963 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 12:20:24.754963 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:24.754963 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:24.754963 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:24.754963 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:24.754963 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:24.754963 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:24.755835 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:24.755835 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:20:24.755835 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:24.755835 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:24.755835 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:24.755835 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:24.755835 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:24.755835 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:24.755835 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:24.755835 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:24.755835 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 12:20:24.755835 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:20:24.755835 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 12:20:24.755835 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:24.760597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 517/1041] 10.222.0.16 () {56 vars in 1105 bytes} [Mon May 23 12:20:24 2022] GET /identity/v3/services/image => generated 86 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:20:24.761163 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0467e5-133b-40da-9cd4-44dd4c7a62d4 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:24.786825 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64662707-b106-484e-8fe6-c00c456d262d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:24.787503 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64662707-b106-484e-8fe6-c00c456d262d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:24.787862 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64662707-b106-484e-8fe6-c00c456d262d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:24.788151 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64662707-b106-484e-8fe6-c00c456d262d None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:24.789025 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64662707-b106-484e-8fe6-c00c456d262d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:24.789349 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64662707-b106-484e-8fe6-c00c456d262d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:24.790778 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64662707-b106-484e-8fe6-c00c456d262d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:24.815112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 525/1042] 10.222.0.16 () {56 vars in 1133 bytes} [Mon May 23 12:20:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:24.821169 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64662707-b106-484e-8fe6-c00c456d262d None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:24.862869 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:24.874636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0467e5-133b-40da-9cd4-44dd4c7a62d4 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:24.911114 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812e34f-843f-4515-8755-ca7dd4bd7990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:24.911781 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812e34f-843f-4515-8755-ca7dd4bd7990 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:24.912073 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812e34f-843f-4515-8755-ca7dd4bd7990 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:24.912384 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812e34f-843f-4515-8755-ca7dd4bd7990 None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:24.913093 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e812e34f-843f-4515-8755-ca7dd4bd7990 None admin] RBAC: Adding query filter params (name=image) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:24.913403 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e812e34f-843f-4515-8755-ca7dd4bd7990 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:24.915194 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e812e34f-843f-4515-8755-ca7dd4bd7990 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:24.920100 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 518/1043] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:20:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:24.925526 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812e34f-843f-4515-8755-ca7dd4bd7990 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:24.931179 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64662707-b106-484e-8fe6-c00c456d262d None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:24.943640 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:24.969835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b8299-bfbb-47d0-831f-91654775697f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:24.970484 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b8299-bfbb-47d0-831f-91654775697f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:24.970770 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b8299-bfbb-47d0-831f-91654775697f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:24.971052 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992b8299-bfbb-47d0-831f-91654775697f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:25.039820 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e812e34f-843f-4515-8755-ca7dd4bd7990 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:25.055838 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b8299-bfbb-47d0-831f-91654775697f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:25.057273 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992b8299-bfbb-47d0-831f-91654775697f None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:25.079934 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937e123-77a5-4972-9624-789e0490c9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:25.080936 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937e123-77a5-4972-9624-789e0490c9cb None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:25.081220 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937e123-77a5-4972-9624-789e0490c9cb None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:25.081500 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937e123-77a5-4972-9624-789e0490c9cb None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:25.082263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937e123-77a5-4972-9624-789e0490c9cb None admin] RBAC: Adding query filter params (type=image) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:25.082668 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937e123-77a5-4972-9624-789e0490c9cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:25.084041 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f937e123-77a5-4972-9624-789e0490c9cb None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:25.093376 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 519/1044] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:20:24 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:25.094339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937e123-77a5-4972-9624-789e0490c9cb None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:25.114814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:25.171220 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 526/1045] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:20:24 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 12:20:25.177485 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b8299-bfbb-47d0-831f-91654775697f None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:25.186543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:25.206357 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f937e123-77a5-4972-9624-789e0490c9cb None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:25.246096 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bfafbe-5dde-4359-af75-4cd747a8ae47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:25.246796 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bfafbe-5dde-4359-af75-4cd747a8ae47 None admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:25.247065 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bfafbe-5dde-4359-af75-4cd747a8ae47 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:25.247317 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bfafbe-5dde-4359-af75-4cd747a8ae47 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:25.248061 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bfafbe-5dde-4359-af75-4cd747a8ae47 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:25.249801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bfafbe-5dde-4359-af75-4cd747a8ae47 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:25.269901 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 520/1046] 10.222.0.16 () {60 vars in 1144 bytes} [Mon May 23 12:20:25 2022] POST /identity/v3/endpoints => generated 327 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:20:25.288506 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992b8299-bfbb-47d0-831f-91654775697f None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:25.336209 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a4383-69c3-4f61-a398-c81a9b0d823c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:25.337132 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a4383-69c3-4f61-a398-c81a9b0d823c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:25.337490 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a4383-69c3-4f61-a398-c81a9b0d823c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:25.337853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497a4383-69c3-4f61-a398-c81a9b0d823c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:25.350154 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a4383-69c3-4f61-a398-c81a9b0d823c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:25.351843 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497a4383-69c3-4f61-a398-c81a9b0d823c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:25.371253 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-497a4383-69c3-4f61-a398-c81a9b0d823c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:20:25.371253 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:25.371253 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:25.371253 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:25.371253 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:25.371253 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:25.371253 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:25.371253 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:25.371253 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:25.371253 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:25.371253 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:25.371253 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:25.371253 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:20:25.371253 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:20:25.371253 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:20:25.372613 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:20:25.372613 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:25.372613 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:25.372613 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:25.372613 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:25.372613 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:25.372613 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:25.372613 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:25.372613 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:20:25.372613 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:25.372613 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:25.372613 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:25.372613 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:25.372613 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:25.372613 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:25.372613 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:25.372613 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:25.373755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:20:25.373755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:20:25.373755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:20:25.373755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:25.374932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 527/1047] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:20:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:20:25.379643 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bfafbe-5dde-4359-af75-4cd747a8ae47 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:25.464578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:25.492590 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bfafbe-5dde-4359-af75-4cd747a8ae47 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:25.526958 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5283d93-c00f-4088-8694-a3d71cf91f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:25.527631 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5283d93-c00f-4088-8694-a3d71cf91f1f None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:25.527919 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5283d93-c00f-4088-8694-a3d71cf91f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:25.528230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5283d93-c00f-4088-8694-a3d71cf91f1f None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:25.528965 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5283d93-c00f-4088-8694-a3d71cf91f1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:25.529275 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5283d93-c00f-4088-8694-a3d71cf91f1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:25.530664 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5283d93-c00f-4088-8694-a3d71cf91f1f None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:25.542608 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 521/1048] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:25.548368 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a4383-69c3-4f61-a398-c81a9b0d823c None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:25.609104 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:25.655766 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497a4383-69c3-4f61-a398-c81a9b0d823c None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:25.690230 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d4907-cc66-4dcc-945d-8a89aee23790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:25.691034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d4907-cc66-4dcc-945d-8a89aee23790 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:25.691982 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d4907-cc66-4dcc-945d-8a89aee23790 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:25.692541 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d4907-cc66-4dcc-945d-8a89aee23790 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:25.699566 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d4907-cc66-4dcc-945d-8a89aee23790 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:25.701030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d4907-cc66-4dcc-945d-8a89aee23790 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:25.714771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-050d4907-cc66-4dcc-945d-8a89aee23790 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:20:25.714771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:25.714771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:25.714771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:25.714771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:25.714771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:25.714771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:25.714771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:25.714771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:25.714771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:25.714771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:25.714771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:25.714771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:20:25.714771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:20:25.714771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:20:25.715658 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:20:25.715658 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:25.715658 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:25.715658 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:25.715658 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:25.715658 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:25.715658 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:25.715658 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:25.715658 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:20:25.715658 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:25.715658 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:25.715658 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:25.715658 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:25.715658 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:25.715658 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:25.715658 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:25.715658 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:25.716739 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:20:25.716739 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:20:25.716739 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:20:25.716739 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:20:25.716739 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:20:25.716739 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:20:25.716739 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:20:25.716739 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:25.716739 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 528/1049] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:20:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:25.721691 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5283d93-c00f-4088-8694-a3d71cf91f1f None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:25.827894 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5283d93-c00f-4088-8694-a3d71cf91f1f None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:25.862872 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa790d-7683-42d4-822c-c14ef94923f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:25.864096 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa790d-7683-42d4-822c-c14ef94923f7 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:25.864408 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa790d-7683-42d4-822c-c14ef94923f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:25.864686 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa790d-7683-42d4-822c-c14ef94923f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:25.865543 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa790d-7683-42d4-822c-c14ef94923f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:25.865858 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa790d-7683-42d4-822c-c14ef94923f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:25.867289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa790d-7683-42d4-822c-c14ef94923f7 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:25.869026 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:25.886071 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 522/1050] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:20:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:25.887951 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d4907-cc66-4dcc-945d-8a89aee23790 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:25.990323 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d4907-cc66-4dcc-945d-8a89aee23790 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:26.009093 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:26.031798 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720677ba-8780-4c32-a08b-afd906818c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:26.032499 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720677ba-8780-4c32-a08b-afd906818c2e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:26.032789 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720677ba-8780-4c32-a08b-afd906818c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:26.033069 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720677ba-8780-4c32-a08b-afd906818c2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:26.033898 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720677ba-8780-4c32-a08b-afd906818c2e None admin] RBAC: Adding query filter params (role.id=3d40411f939f4ff6af2e78c756f6cc9e, scope.project.id=839585501f414e1ab02501422510e54f, user.id=b83535b443a84275a160cd84cb3ad5a0) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:26.034217 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720677ba-8780-4c32-a08b-afd906818c2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:26.036303 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720677ba-8780-4c32-a08b-afd906818c2e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:26.048427 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 529/1051] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:20:25 2022] GET /identity/v3/role_assignments?user.id=b83535b443a84275a160cd84cb3ad5a0&scope.project.id=839585501f414e1ab02501422510e54f&role.id=3d40411f939f4ff6af2e78c756f6cc9e => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:26.189537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922f0a7-6165-456d-ac57-3ff2465f8f93 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:26.189896 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922f0a7-6165-456d-ac57-3ff2465f8f93 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:26.190106 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922f0a7-6165-456d-ac57-3ff2465f8f93 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:26.191734 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 523/1052] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:26.197816 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b2f33-c80a-4c0e-9f8c-f3d6a0c272d9 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:26.198114 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b2f33-c80a-4c0e-9f8c-f3d6a0c272d9 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:26.198394 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b2f33-c80a-4c0e-9f8c-f3d6a0c272d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:26.216576 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-141b2f33-c80a-4c0e-9f8c-f3d6a0c272d9 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:26.217775 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-141b2f33-c80a-4c0e-9f8c-f3d6a0c272d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:26.290307 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 530/1053] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:20:26.313081 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922f0a7-6165-456d-ac57-3ff2465f8f93 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:26.418148 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922f0a7-6165-456d-ac57-3ff2465f8f93 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:26.455746 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d1aeb-077d-4b05-bbf7-893da4e14064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:26.456554 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d1aeb-077d-4b05-bbf7-893da4e14064 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:26.456873 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d1aeb-077d-4b05-bbf7-893da4e14064 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:26.457344 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d1aeb-077d-4b05-bbf7-893da4e14064 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:26.467841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d1aeb-077d-4b05-bbf7-893da4e14064 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:20:26.476376 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-603d1aeb-077d-4b05-bbf7-893da4e14064 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:20:26.476376 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:26.476376 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:26.476376 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:26.476376 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:26.476376 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:26.476376 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:26.476376 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:26.476376 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:26.476376 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:26.476376 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:26.476376 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:26.476376 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:20:26.476376 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:20:26.476376 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:20:26.476376 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:20:26.477332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:20:26.477332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:20:26.477332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:20:26.477332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:20:26.477332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:26.477332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:26.477332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:26.477332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:26.477332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:26.477332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:26.477332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:26.477332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:20:26.477332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:26.477332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:26.477332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:26.477332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:26.478215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:26.478215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:26.478215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:26.478215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:26.478215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:20:26.478215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:20:26.478215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:20:26.478215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:20:26.478215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:20:26.478215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:20:26.478215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:20:26.478215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:26.482503 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 524/1054] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:20:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:26.483483 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b2f33-c80a-4c0e-9f8c-f3d6a0c272d9 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:26.585961 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b2f33-c80a-4c0e-9f8c-f3d6a0c272d9 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:26.596628 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:26.624181 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c2361-7bc0-48f2-9616-b6065d54d493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:26.624878 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c2361-7bc0-48f2-9616-b6065d54d493 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:26.625160 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c2361-7bc0-48f2-9616-b6065d54d493 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:26.625437 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c2361-7bc0-48f2-9616-b6065d54d493 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:26.626179 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c2361-7bc0-48f2-9616-b6065d54d493 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:26.626485 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c2361-7bc0-48f2-9616-b6065d54d493 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:26.627834 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c2361-7bc0-48f2-9616-b6065d54d493 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:26.640955 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 531/1055] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:20:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:26.642206 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d1aeb-077d-4b05-bbf7-893da4e14064 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:26.701272 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:26.747952 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d1aeb-077d-4b05-bbf7-893da4e14064 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:26.784699 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e9c3b-b258-4065-98a0-2c474fa16f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:26.785778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e9c3b-b258-4065-98a0-2c474fa16f0f None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:26.786062 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e9c3b-b258-4065-98a0-2c474fa16f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:26.786344 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e9c3b-b258-4065-98a0-2c474fa16f0f None admin] PATH_INFO: `/v3/users/7630f9921f104c57b14fc0bf3e0b20ad` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:26.800122 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e9c3b-b258-4065-98a0-2c474fa16f0f None admin] RBAC: Authorizing `identity:get_user(user_id=7630f9921f104c57b14fc0bf3e0b20ad)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:26.801675 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e9c3b-b258-4065-98a0-2c474fa16f0f None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:26.803571 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 525/1056] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:20:26 2022] GET /identity/v3/users/7630f9921f104c57b14fc0bf3e0b20ad => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:26.809103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c2361-7bc0-48f2-9616-b6065d54d493 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:26.915850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c2361-7bc0-48f2-9616-b6065d54d493 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:26.954603 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7732076-66c3-462f-a28e-4e41e1cf4f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:26.955257 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7732076-66c3-462f-a28e-4e41e1cf4f16 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:26.955548 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7732076-66c3-462f-a28e-4e41e1cf4f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:26.955830 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7732076-66c3-462f-a28e-4e41e1cf4f16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:26.998871 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:27.034015 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7732076-66c3-462f-a28e-4e41e1cf4f16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:27.035652 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7732076-66c3-462f-a28e-4e41e1cf4f16 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:27.144210 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 532/1057] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:20:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 12:20:27.145840 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e9c3b-b258-4065-98a0-2c474fa16f0f None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:27.247502 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e9c3b-b258-4065-98a0-2c474fa16f0f None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:27.261912 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:27.283942 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce6e989-3cd9-44e0-a2e5-0bc507efbc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:27.284825 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce6e989-3cd9-44e0-a2e5-0bc507efbc89 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:27.285766 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce6e989-3cd9-44e0-a2e5-0bc507efbc89 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:27.286215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce6e989-3cd9-44e0-a2e5-0bc507efbc89 None admin] PATH_INFO: `/v3/projects/c6d0602ebd9c4e1083e8d05e49f38548` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:27.287389 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce6e989-3cd9-44e0-a2e5-0bc507efbc89 None admin] RBAC: Authorizing `identity:get_project(project_id=c6d0602ebd9c4e1083e8d05e49f38548)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:27.288856 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce6e989-3cd9-44e0-a2e5-0bc507efbc89 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:27.290682 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 526/1058] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:20:27 2022] GET /identity/v3/projects/c6d0602ebd9c4e1083e8d05e49f38548 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:27.296487 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7732076-66c3-462f-a28e-4e41e1cf4f16 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:27.373167 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:27.403792 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7732076-66c3-462f-a28e-4e41e1cf4f16 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:27.438244 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ed298-4dfc-43e5-8bc2-d4b59671869c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:27.438889 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ed298-4dfc-43e5-8bc2-d4b59671869c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:27.439170 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ed298-4dfc-43e5-8bc2-d4b59671869c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:27.439441 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ed298-4dfc-43e5-8bc2-d4b59671869c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:27.440278 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ed298-4dfc-43e5-8bc2-d4b59671869c None admin] RBAC: Adding query filter params (role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8, scope.project.id=c6d0602ebd9c4e1083e8d05e49f38548, user.id=7630f9921f104c57b14fc0bf3e0b20ad) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:27.440591 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ed298-4dfc-43e5-8bc2-d4b59671869c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:27.442616 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85ed298-4dfc-43e5-8bc2-d4b59671869c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:27.448157 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 533/1059] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:20:27 2022] GET /identity/v3/role_assignments?user.id=7630f9921f104c57b14fc0bf3e0b20ad&scope.project.id=c6d0602ebd9c4e1083e8d05e49f38548&role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:27.525236 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6be4bb-3cf9-41fb-91d1-456b9f0f6e29 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:27.525775 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6be4bb-3cf9-41fb-91d1-456b9f0f6e29 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:27.525775 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6be4bb-3cf9-41fb-91d1-456b9f0f6e29 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:27.527240 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 527/1060] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:27.532971 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d007a-e658-4d95-8589-e55c1b6b8402 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:27.533274 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d007a-e658-4d95-8589-e55c1b6b8402 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:27.533551 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d007a-e658-4d95-8589-e55c1b6b8402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:27.553349 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-753d007a-e658-4d95-8589-e55c1b6b8402 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:27.554665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-753d007a-e658-4d95-8589-e55c1b6b8402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:27.632225 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 534/1061] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:20:27.652451 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6be4bb-3cf9-41fb-91d1-456b9f0f6e29 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:27.757340 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6be4bb-3cf9-41fb-91d1-456b9f0f6e29 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:27.794440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f09b3-5dbf-45fe-ad29-a0d4eb01b37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:27.795010 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f09b3-5dbf-45fe-ad29-a0d4eb01b37e None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:27.795290 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f09b3-5dbf-45fe-ad29-a0d4eb01b37e None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:27.795566 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f09b3-5dbf-45fe-ad29-a0d4eb01b37e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:27.844709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:27.865491 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f09b3-5dbf-45fe-ad29-a0d4eb01b37e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:27.867109 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f09b3-5dbf-45fe-ad29-a0d4eb01b37e None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:27.970919 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 528/1062] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:20:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 12:20:27.973848 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d007a-e658-4d95-8589-e55c1b6b8402 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:28.048418 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:28.087589 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d007a-e658-4d95-8589-e55c1b6b8402 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:28.121854 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c0714-fcc6-4154-9aca-479ce787d77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:28.122647 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c0714-fcc6-4154-9aca-479ce787d77a None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:28.122931 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c0714-fcc6-4154-9aca-479ce787d77a None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:28.123208 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c0714-fcc6-4154-9aca-479ce787d77a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:28.134167 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c0714-fcc6-4154-9aca-479ce787d77a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:28.135612 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c0714-fcc6-4154-9aca-479ce787d77a None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:28.153477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-221c0714-fcc6-4154-9aca-479ce787d77a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:20:28.153477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:28.153477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:28.153477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:28.153477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:28.153477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:28.153477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:28.153477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:28.153477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:28.153477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:28.153477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:28.153477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:28.153477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:20:28.153477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:20:28.153477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:20:28.154401 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:20:28.154401 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:28.154401 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:28.154401 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:28.154401 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:28.154401 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:28.154401 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:28.154401 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:28.154401 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:20:28.154401 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:28.154401 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:28.154401 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:28.154401 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:28.154401 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:28.154401 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:28.154401 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:28.154401 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:28.155360 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:20:28.155360 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:20:28.155360 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:20:28.155360 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:28.159645 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 535/1063] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:20:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:20:28.161304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f09b3-5dbf-45fe-ad29-a0d4eb01b37e None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:28.235130 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248c87e-ecb7-4efa-aec5-ea0ff6996619 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:28.235472 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248c87e-ecb7-4efa-aec5-ea0ff6996619 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:28.235709 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248c87e-ecb7-4efa-aec5-ea0ff6996619 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:28.237240 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 536/1064] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:20:28.242337 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7829a-4c38-416d-abd7-c165334a1342 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:28.242629 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7829a-4c38-416d-abd7-c165334a1342 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:28.242908 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7829a-4c38-416d-abd7-c165334a1342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:28.246437 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:28.260596 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-d1b7829a-4c38-416d-abd7-c165334a1342 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:28.261799 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d1b7829a-4c38-416d-abd7-c165334a1342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:28.270920 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f09b3-5dbf-45fe-ad29-a0d4eb01b37e None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:28.307803 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393311ed-8536-4cc6-ba30-fdf215bde2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:28.308422 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393311ed-8536-4cc6-ba30-fdf215bde2b3 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:28.308718 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393311ed-8536-4cc6-ba30-fdf215bde2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:28.309004 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393311ed-8536-4cc6-ba30-fdf215bde2b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:28.309752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393311ed-8536-4cc6-ba30-fdf215bde2b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:28.310044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393311ed-8536-4cc6-ba30-fdf215bde2b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:28.311421 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393311ed-8536-4cc6-ba30-fdf215bde2b3 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:28.323634 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 529/1065] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:28.332479 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 537/1066] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 12:20:28.354821 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393311ed-8536-4cc6-ba30-fdf215bde2b3 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:28.466851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393311ed-8536-4cc6-ba30-fdf215bde2b3 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:28.502380 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07260724-3f43-4690-a8a1-e21ba8ce27a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:28.503200 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07260724-3f43-4690-a8a1-e21ba8ce27a7 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:28.503488 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07260724-3f43-4690-a8a1-e21ba8ce27a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:28.504349 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07260724-3f43-4690-a8a1-e21ba8ce27a7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:28.512782 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:28.516854 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07260724-3f43-4690-a8a1-e21ba8ce27a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:20:28.525559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-07260724-3f43-4690-a8a1-e21ba8ce27a7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 12:20:28.525559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:28.525559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:28.525559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:28.525559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:28.525559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:28.525559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:28.525559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:28.525559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:28.525559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:28.525559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:28.525559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:28.525559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:20:28.525559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:20:28.525559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:20:28.525559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:20:28.526924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:20:28.526924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:20:28.526924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:20:28.526924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:20:28.526924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:28.526924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:28.526924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:28.526924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:28.526924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:28.526924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:28.526924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:28.526924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:20:28.526924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:28.526924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:28.526924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:28.526924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:28.528243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:28.528243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:28.528243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:28.528243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:28.528243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:20:28.528243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:20:28.528243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:20:28.528243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:20:28.528243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:20:28.528243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:20:28.528243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 12:20:28.528243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:28.528243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 530/1067] 10.222.0.16 () {56 vars in 1120 bytes} [Mon May 23 12:20:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:28.532948 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7829a-4c38-416d-abd7-c165334a1342 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:28.591022 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:28.649539 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7829a-4c38-416d-abd7-c165334a1342 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:28.686320 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c55ca0-3a05-451d-a952-e9a8fe0ab13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:28.686996 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c55ca0-3a05-451d-a952-e9a8fe0ab13d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:28.687282 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c55ca0-3a05-451d-a952-e9a8fe0ab13d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:28.687559 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c55ca0-3a05-451d-a952-e9a8fe0ab13d None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:28.688332 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c55ca0-3a05-451d-a952-e9a8fe0ab13d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:28.688644 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c55ca0-3a05-451d-a952-e9a8fe0ab13d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:28.690004 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c55ca0-3a05-451d-a952-e9a8fe0ab13d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:28.699118 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 538/1068] 10.222.0.16 () {56 vars in 1115 bytes} [Mon May 23 12:20:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:28.705203 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07260724-3f43-4690-a8a1-e21ba8ce27a7 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:28.805520 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07260724-3f43-4690-a8a1-e21ba8ce27a7 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:28.839033 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29d034-2a67-41cf-bb02-89f4a5ee1db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:28.839811 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29d034-2a67-41cf-bb02-89f4a5ee1db7 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:28.840094 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29d034-2a67-41cf-bb02-89f4a5ee1db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:28.840402 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29d034-2a67-41cf-bb02-89f4a5ee1db7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:28.850701 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf29d034-2a67-41cf-bb02-89f4a5ee1db7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:28.852086 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf29d034-2a67-41cf-bb02-89f4a5ee1db7 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:28.869476 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-bf29d034-2a67-41cf-bb02-89f4a5ee1db7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:20:28.869476 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:28.869476 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:28.869476 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:28.869476 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:28.869476 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:28.869476 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:28.869476 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:28.869476 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:28.869476 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:28.869476 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:28.869476 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:28.869476 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:20:28.869476 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:20:28.869476 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:20:28.870363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:20:28.870363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:28.870363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:28.870363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:28.870363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:28.870363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:28.870363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:28.870363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:28.870363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:20:28.870363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:28.870363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:28.870363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:28.870363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:28.870363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:28.870363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:28.870363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:28.870363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:28.871295 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:20:28.871295 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:20:28.871295 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:20:28.871295 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:28.871295 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 531/1069] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:20:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:20:28.876384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c55ca0-3a05-451d-a952-e9a8fe0ab13d None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:28.974117 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c55ca0-3a05-451d-a952-e9a8fe0ab13d None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:28.984584 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:29.008679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b1a22-f417-41b0-b0cc-4589290ac7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:29.009338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b1a22-f417-41b0-b0cc-4589290ac7bf None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:29.009620 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b1a22-f417-41b0-b0cc-4589290ac7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:29.009896 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b1a22-f417-41b0-b0cc-4589290ac7bf None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:29.010607 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7b1a22-f417-41b0-b0cc-4589290ac7bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:29.010911 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7b1a22-f417-41b0-b0cc-4589290ac7bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:29.012303 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7b1a22-f417-41b0-b0cc-4589290ac7bf None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:29.027400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 539/1070] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:29.032860 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29d034-2a67-41cf-bb02-89f4a5ee1db7 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:29.111921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:29.162952 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29d034-2a67-41cf-bb02-89f4a5ee1db7 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:29.197098 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805313a-10f9-42b4-989f-514fab6cf310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:29.197881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805313a-10f9-42b4-989f-514fab6cf310 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:29.198162 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805313a-10f9-42b4-989f-514fab6cf310 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:29.198436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805313a-10f9-42b4-989f-514fab6cf310 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:29.205481 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0805313a-10f9-42b4-989f-514fab6cf310 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:29.207088 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0805313a-10f9-42b4-989f-514fab6cf310 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:29.221230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-0805313a-10f9-42b4-989f-514fab6cf310 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 12:20:29.221230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:29.221230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:29.221230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:29.221230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:29.221230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:29.221230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:29.221230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:29.221230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:29.221230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:29.221230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:29.221230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:29.221230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:20:29.221230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:20:29.221230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:20:29.221230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:20:29.222360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:29.222360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:29.222360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:20:29.222360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:20:29.222360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:20:29.222360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:20:29.222360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:29.222360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:29.222360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:29.222360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:29.222360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:29.222360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:20:29.222360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:29.222360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:29.222360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:29.222360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:29.222360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:29.222360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:29.223487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:29.223487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:29.223487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:20:29.223487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:20:29.223487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:20:29.223487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:20:29.223487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:20:29.223487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:20:29.223487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 12:20:29.223487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:29.223487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 532/1071] 10.222.0.16 () {56 vars in 1093 bytes} [Mon May 23 12:20:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:29.228372 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b1a22-f417-41b0-b0cc-4589290ac7bf None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:29.330347 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b1a22-f417-41b0-b0cc-4589290ac7bf None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:29.366264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7e060-5b70-4641-b3e6-ab93f36c2dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:29.366853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7e060-5b70-4641-b3e6-ab93f36c2dae None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:29.367139 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7e060-5b70-4641-b3e6-ab93f36c2dae None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:29.367415 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7e060-5b70-4641-b3e6-ab93f36c2dae None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:29.368270 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7e060-5b70-4641-b3e6-ab93f36c2dae None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:29.368582 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7e060-5b70-4641-b3e6-ab93f36c2dae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:29.370003 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7e060-5b70-4641-b3e6-ab93f36c2dae None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:29.388350 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 540/1072] 10.222.0.16 () {56 vars in 1133 bytes} [Mon May 23 12:20:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:29.393808 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805313a-10f9-42b4-989f-514fab6cf310 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:29.505946 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805313a-10f9-42b4-989f-514fab6cf310 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:29.520757 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:29.542580 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f44c5-1cfc-44db-9b4a-04aba4446363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:29.543368 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f44c5-1cfc-44db-9b4a-04aba4446363 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:29.543653 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f44c5-1cfc-44db-9b4a-04aba4446363 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:29.543934 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f44c5-1cfc-44db-9b4a-04aba4446363 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:29.554647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f44c5-1cfc-44db-9b4a-04aba4446363 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:29.556101 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f44c5-1cfc-44db-9b4a-04aba4446363 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:29.573592 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-a33f44c5-1cfc-44db-9b4a-04aba4446363 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:20:29.573592 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:29.573592 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:29.573592 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:29.573592 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:29.573592 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:29.573592 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:29.573592 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:29.573592 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:29.573592 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:29.573592 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:29.573592 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:29.573592 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:20:29.573592 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:20:29.573592 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:20:29.574534 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:20:29.574534 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:29.574534 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:29.574534 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:29.574534 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:29.574534 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:29.574534 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:29.574534 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:29.574534 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:20:29.574534 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:29.574534 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:29.574534 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:29.574534 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:29.574534 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:29.574534 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:29.574534 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:29.574534 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:29.575557 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:20:29.575557 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:20:29.575557 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:20:29.575557 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:29.575557 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 533/1073] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:20:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:29.580374 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7e060-5b70-4641-b3e6-ab93f36c2dae None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:29.609786 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:29.682078 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7e060-5b70-4641-b3e6-ab93f36c2dae None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:29.684497 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310677a0-0f34-4f1b-9293-a5fdfe926a0b None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:29.684793 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310677a0-0f34-4f1b-9293-a5fdfe926a0b None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:29.685076 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310677a0-0f34-4f1b-9293-a5fdfe926a0b None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:29.686416 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 534/1074] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:20:29.692240 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0adf28-fc6a-4edd-92e2-c70f0223e7aa None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:29.692788 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0adf28-fc6a-4edd-92e2-c70f0223e7aa None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:29.693071 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0adf28-fc6a-4edd-92e2-c70f0223e7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:29.709918 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-0e0adf28-fc6a-4edd-92e2-c70f0223e7aa None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:29.711107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-0e0adf28-fc6a-4edd-92e2-c70f0223e7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:29.716873 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d9723-728d-4247-b389-339046eb8df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:29.717529 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d9723-728d-4247-b389-339046eb8df9 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:29.717813 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d9723-728d-4247-b389-339046eb8df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:29.718088 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d9723-728d-4247-b389-339046eb8df9 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:29.718811 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d9723-728d-4247-b389-339046eb8df9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:29.719114 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d9723-728d-4247-b389-339046eb8df9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:29.720505 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d9723-728d-4247-b389-339046eb8df9 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:29.731781 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 541/1075] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:29.737361 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d9723-728d-4247-b389-339046eb8df9 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:29.778504 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 535/1076] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 12:20:29.800096 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0adf28-fc6a-4edd-92e2-c70f0223e7aa None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:29.824691 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:29.837337 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d9723-728d-4247-b389-339046eb8df9 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:29.870529 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa89c3a-63e1-440e-8943-25273f08afab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:29.871326 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa89c3a-63e1-440e-8943-25273f08afab None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:29.871613 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa89c3a-63e1-440e-8943-25273f08afab None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:29.871896 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa89c3a-63e1-440e-8943-25273f08afab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:29.878582 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa89c3a-63e1-440e-8943-25273f08afab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:29.880164 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa89c3a-63e1-440e-8943-25273f08afab None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:29.890758 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:29.895032 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-8fa89c3a-63e1-440e-8943-25273f08afab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:20:29.895032 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:29.895032 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:29.895032 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:29.895032 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:29.895032 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:29.895032 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:29.895032 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:29.895032 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:29.895032 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:29.895032 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:29.895032 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:29.895032 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:20:29.895032 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:20:29.895032 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:20:29.895943 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:20:29.895943 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:29.895943 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:29.895943 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:29.895943 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:29.895943 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:29.895943 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:29.895943 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:29.895943 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:20:29.895943 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:29.895943 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:29.895943 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:29.895943 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:29.895943 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:29.895943 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:29.895943 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:29.895943 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:29.896896 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:20:29.896896 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:20:29.896896 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:20:29.896896 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:20:29.896896 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:20:29.896896 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:20:29.896896 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:20:29.896896 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:29.896896 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 542/1077] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:20:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:20:29.900465 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0adf28-fc6a-4edd-92e2-c70f0223e7aa None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:29.902024 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa89c3a-63e1-440e-8943-25273f08afab None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:29.935260 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986884fa-3447-4466-945b-930ecc5eaee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:29.935980 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986884fa-3447-4466-945b-930ecc5eaee2 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:29.936295 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986884fa-3447-4466-945b-930ecc5eaee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:29.936579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986884fa-3447-4466-945b-930ecc5eaee2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:29.947485 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986884fa-3447-4466-945b-930ecc5eaee2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:20:29.955486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-986884fa-3447-4466-945b-930ecc5eaee2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:20:29.955486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:29.955486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:29.955486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:29.955486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:29.955486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:29.955486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:29.955486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:29.955486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:29.955486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:29.955486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:29.955486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:29.955486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:20:29.955486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:20:29.955486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:20:29.955486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:20:29.956462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:20:29.956462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:20:29.956462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:20:29.956462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:20:29.956462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:29.956462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:29.956462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:29.956462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:29.956462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:29.956462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:29.956462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:29.956462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:20:29.956462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:29.956462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:29.956462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:29.956462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:29.957872 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:29.957872 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:29.957872 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:29.957872 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:29.957872 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:20:29.957872 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:20:29.957872 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:20:29.957872 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:20:29.957872 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:20:29.957872 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:20:29.957872 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:20:29.957872 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:29.957872 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 536/1078] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:20:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:20:29.963087 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986884fa-3447-4466-945b-930ecc5eaee2 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:30.000733 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa89c3a-63e1-440e-8943-25273f08afab None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:30.038099 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd8b82e-527d-432a-ad09-aa848a454e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:30.038754 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8b82e-527d-432a-ad09-aa848a454e4a None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:30.039037 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8b82e-527d-432a-ad09-aa848a454e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:30.039314 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8b82e-527d-432a-ad09-aa848a454e4a None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:30.040090 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd8b82e-527d-432a-ad09-aa848a454e4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:30.040435 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd8b82e-527d-432a-ad09-aa848a454e4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:30.041846 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd8b82e-527d-432a-ad09-aa848a454e4a None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:30.053180 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 543/1079] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:20:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:30.058951 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd8b82e-527d-432a-ad09-aa848a454e4a None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:30.066065 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986884fa-3447-4466-945b-930ecc5eaee2 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:30.070200 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:30.100600 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b288d13-7855-4bb2-b37e-fb9c492b1ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:30.101258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b288d13-7855-4bb2-b37e-fb9c492b1ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:30.101537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b288d13-7855-4bb2-b37e-fb9c492b1ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:30.101811 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b288d13-7855-4bb2-b37e-fb9c492b1ac3 None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:30.102554 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b288d13-7855-4bb2-b37e-fb9c492b1ac3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:30.102857 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b288d13-7855-4bb2-b37e-fb9c492b1ac3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:30.104259 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b288d13-7855-4bb2-b37e-fb9c492b1ac3 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:30.112386 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 537/1080] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:20:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:30.118318 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b288d13-7855-4bb2-b37e-fb9c492b1ac3 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:30.135940 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:30.164882 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd8b82e-527d-432a-ad09-aa848a454e4a None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:30.199162 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ce3a1-034c-45fd-a311-be4eb5bba934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:30.199980 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ce3a1-034c-45fd-a311-be4eb5bba934 None admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:30.200293 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ce3a1-034c-45fd-a311-be4eb5bba934 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:30.200578 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ce3a1-034c-45fd-a311-be4eb5bba934 None admin] PATH_INFO: `/v3/projects/839585501f414e1ab02501422510e54f/users/b83535b443a84275a160cd84cb3ad5a0/roles/3d40411f939f4ff6af2e78c756f6cc9e` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:30.202784 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ce3a1-034c-45fd-a311-be4eb5bba934 None admin] RBAC: Authorizing `identity:create_grant(project_id=839585501f414e1ab02501422510e54f, user_id=b83535b443a84275a160cd84cb3ad5a0, role_id=3d40411f939f4ff6af2e78c756f6cc9e)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:30.204243 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ce3a1-034c-45fd-a311-be4eb5bba934 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:30.217889 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 544/1081] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 12:20:30 2022] PUT /identity/v3/projects/839585501f414e1ab02501422510e54f/users/b83535b443a84275a160cd84cb3ad5a0/roles/3d40411f939f4ff6af2e78c756f6cc9e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:20:30.221689 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b288d13-7855-4bb2-b37e-fb9c492b1ac3 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:30.259480 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46910168-d49f-421b-b39e-9a1bbc982c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:30.260521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46910168-d49f-421b-b39e-9a1bbc982c96 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:30.260806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46910168-d49f-421b-b39e-9a1bbc982c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:30.261105 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46910168-d49f-421b-b39e-9a1bbc982c96 None admin] PATH_INFO: `/v3/users/7630f9921f104c57b14fc0bf3e0b20ad` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:30.262402 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46910168-d49f-421b-b39e-9a1bbc982c96 None admin] RBAC: Authorizing `identity:get_user(user_id=7630f9921f104c57b14fc0bf3e0b20ad)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:30.263907 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46910168-d49f-421b-b39e-9a1bbc982c96 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:30.265641 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 538/1082] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:20:30 2022] GET /identity/v3/users/7630f9921f104c57b14fc0bf3e0b20ad => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:30.272075 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ce3a1-034c-45fd-a311-be4eb5bba934 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:30.342240 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:30.378507 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ce3a1-034c-45fd-a311-be4eb5bba934 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:30.415448 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2f431-8466-48c2-b4f9-1b5f7cad57d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:30.416345 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2f431-8466-48c2-b4f9-1b5f7cad57d5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:30.416626 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2f431-8466-48c2-b4f9-1b5f7cad57d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:30.416904 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2f431-8466-48c2-b4f9-1b5f7cad57d5 None admin] PATH_INFO: `/v3/projects/c6d0602ebd9c4e1083e8d05e49f38548` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:30.418063 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2f431-8466-48c2-b4f9-1b5f7cad57d5 None admin] RBAC: Authorizing `identity:get_project(project_id=c6d0602ebd9c4e1083e8d05e49f38548)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:30.419485 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2f431-8466-48c2-b4f9-1b5f7cad57d5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:30.421225 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 545/1083] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:20:30 2022] GET /identity/v3/projects/c6d0602ebd9c4e1083e8d05e49f38548 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:30.427580 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46910168-d49f-421b-b39e-9a1bbc982c96 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:30.532868 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:30.550355 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46910168-d49f-421b-b39e-9a1bbc982c96 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:30.584449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff5c59-3646-4f5f-a2cb-5dbbbbd6cd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:30.585342 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff5c59-3646-4f5f-a2cb-5dbbbbd6cd19 None admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:30.585629 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff5c59-3646-4f5f-a2cb-5dbbbbd6cd19 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:30.585913 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff5c59-3646-4f5f-a2cb-5dbbbbd6cd19 None admin] PATH_INFO: `/v3/projects/c6d0602ebd9c4e1083e8d05e49f38548/users/7630f9921f104c57b14fc0bf3e0b20ad/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:30.587873 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff5c59-3646-4f5f-a2cb-5dbbbbd6cd19 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6d0602ebd9c4e1083e8d05e49f38548, user_id=7630f9921f104c57b14fc0bf3e0b20ad, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:30.589359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff5c59-3646-4f5f-a2cb-5dbbbbd6cd19 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:30.602035 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 539/1084] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 12:20:30 2022] PUT /identity/v3/projects/c6d0602ebd9c4e1083e8d05e49f38548/users/7630f9921f104c57b14fc0bf3e0b20ad/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:20:32.430256 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21a77b-e92d-488a-99ea-1e82c327c34f None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:32.431918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21a77b-e92d-488a-99ea-1e82c327c34f None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:32.433386 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21a77b-e92d-488a-99ea-1e82c327c34f None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:32.434279 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 546/1085] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:32.439674 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b63385-920f-45c2-a37d-c3ac7ba8a856 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:32.440117 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b63385-920f-45c2-a37d-c3ac7ba8a856 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:32.440256 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b63385-920f-45c2-a37d-c3ac7ba8a856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:32.458073 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-11b63385-920f-45c2-a37d-c3ac7ba8a856 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:32.459234 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-11b63385-920f-45c2-a37d-c3ac7ba8a856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:32.531593 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 540/1086] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:20:32.556061 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21a77b-e92d-488a-99ea-1e82c327c34f None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:32.661748 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21a77b-e92d-488a-99ea-1e82c327c34f None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:32.695888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08555de-42ad-4b26-89a2-86f33a97d16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:32.696662 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08555de-42ad-4b26-89a2-86f33a97d16f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:32.696943 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08555de-42ad-4b26-89a2-86f33a97d16f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:32.697222 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08555de-42ad-4b26-89a2-86f33a97d16f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:32.699780 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:32.708359 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08555de-42ad-4b26-89a2-86f33a97d16f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:20:32.716384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-f08555de-42ad-4b26-89a2-86f33a97d16f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 12:20:32.716384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:32.716384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:32.716384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:32.716384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:32.716384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:32.716384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:32.716384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:32.716384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:32.716384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:32.716384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:32.716384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:32.716384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:20:32.716384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:20:32.716384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:20:32.716384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:20:32.717555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:20:32.717555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:20:32.717555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:20:32.717555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:20:32.717555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:32.717555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:32.717555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:32.717555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:32.717555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:32.717555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:32.717555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:32.717555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:20:32.717555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:32.717555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:32.717555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:32.717555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:32.718778 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:32.718778 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:32.718778 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:32.718778 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:32.718778 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:20:32.718778 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:20:32.718778 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:20:32.718778 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:20:32.718778 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:20:32.718778 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:20:32.718778 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 12:20:32.718778 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:32.718778 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 547/1087] 10.222.0.16 () {56 vars in 1120 bytes} [Mon May 23 12:20:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:32.724043 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b63385-920f-45c2-a37d-c3ac7ba8a856 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:32.791405 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3dc239-332e-4295-8c11-20200554c6dd None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:32.791713 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3dc239-332e-4295-8c11-20200554c6dd None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:32.791995 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3dc239-332e-4295-8c11-20200554c6dd None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:32.793383 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 548/1088] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:20:32.798465 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279daea9-94f3-4470-966b-342e8bc3e434 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:32.798751 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279daea9-94f3-4470-966b-342e8bc3e434 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:32.799029 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279daea9-94f3-4470-966b-342e8bc3e434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:32.801965 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:32.819703 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-279daea9-94f3-4470-966b-342e8bc3e434 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:32.820935 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-279daea9-94f3-4470-966b-342e8bc3e434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:32.834113 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b63385-920f-45c2-a37d-c3ac7ba8a856 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:32.867912 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33442fa4-bdc0-47be-a808-414efb698dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:32.868604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33442fa4-bdc0-47be-a808-414efb698dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:32.868891 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33442fa4-bdc0-47be-a808-414efb698dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:32.869166 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33442fa4-bdc0-47be-a808-414efb698dd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:32.869915 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33442fa4-bdc0-47be-a808-414efb698dd9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:32.870225 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33442fa4-bdc0-47be-a808-414efb698dd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:32.871592 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33442fa4-bdc0-47be-a808-414efb698dd9 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:32.879495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 541/1089] 10.222.0.16 () {56 vars in 1115 bytes} [Mon May 23 12:20:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:32.885090 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33442fa4-bdc0-47be-a808-414efb698dd9 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:32.887806 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 549/1090] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 12:20:32.912070 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279daea9-94f3-4470-966b-342e8bc3e434 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:32.989663 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33442fa4-bdc0-47be-a808-414efb698dd9 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:33.008606 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279daea9-94f3-4470-966b-342e8bc3e434 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:33.028304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87aa65-cd74-4ca3-9a6b-e14da0fc8b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:33.029110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87aa65-cd74-4ca3-9a6b-e14da0fc8b70 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:33.029824 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87aa65-cd74-4ca3-9a6b-e14da0fc8b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:33.030112 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87aa65-cd74-4ca3-9a6b-e14da0fc8b70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:33.033083 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:33.035619 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:33.043315 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b87aa65-cd74-4ca3-9a6b-e14da0fc8b70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:33.044806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b87aa65-cd74-4ca3-9a6b-e14da0fc8b70 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:33.045490 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f813e-44b4-4882-8c53-e4f6082a95ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:33.046278 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f813e-44b4-4882-8c53-e4f6082a95ea None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:33.046561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f813e-44b4-4882-8c53-e4f6082a95ea None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:33.046834 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f813e-44b4-4882-8c53-e4f6082a95ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:33.056637 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f813e-44b4-4882-8c53-e4f6082a95ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:20:33.061511 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-8b87aa65-cd74-4ca3-9a6b-e14da0fc8b70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:20:33.061511 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:33.061511 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:33.061511 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:33.061511 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:33.061511 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:33.061511 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:33.061511 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:33.061511 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:33.061511 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:33.061511 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:33.061511 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:33.061511 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:20:33.061511 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:20:33.061511 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:20:33.062547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:20:33.062547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:33.062547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:33.062547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:33.062547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:33.062547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:33.062547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:33.062547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:33.062547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:20:33.062547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:33.062547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:33.062547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:33.062547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:33.062547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:33.062547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:33.062547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:33.062547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:33.068798 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:20:33.068798 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:20:33.068798 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:20:33.068798 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:33.068798 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 542/1091] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:20:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:20:33.069122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-516f813e-44b4-4882-8c53-e4f6082a95ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:20:33.069122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:33.069122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:33.069122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:33.069122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:33.069122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:33.069122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:33.069122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:33.069122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:33.069122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:33.069122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:33.069122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:33.069122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:20:33.069122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:20:33.069122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:20:33.069122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:20:33.069981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:20:33.069981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:20:33.069981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:20:33.069981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:20:33.069981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:33.069981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:33.069981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:33.069981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:33.069981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:33.069981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:33.069981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:33.069981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:20:33.069981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:33.069981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:33.069981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:33.069981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:33.070839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:33.070839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:33.070839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:33.070839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:33.070839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:20:33.070839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:20:33.070839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:20:33.070839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:20:33.070839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:20:33.070839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:20:33.070839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:20:33.070839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:33.070839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 550/1092] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:20:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:20:33.071552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f813e-44b4-4882-8c53-e4f6082a95ea None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:33.071786 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87aa65-cd74-4ca3-9a6b-e14da0fc8b70 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:33.168963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f813e-44b4-4882-8c53-e4f6082a95ea None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:33.170851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87aa65-cd74-4ca3-9a6b-e14da0fc8b70 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:33.202535 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc8927-ec0d-4223-b85f-e4171018e50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:33.203204 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc8927-ec0d-4223-b85f-e4171018e50d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:33.203495 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc8927-ec0d-4223-b85f-e4171018e50d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:33.203614 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319bcbf-f0f4-47b6-b3c6-1b510cc42adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:33.203773 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc8927-ec0d-4223-b85f-e4171018e50d None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:33.204208 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319bcbf-f0f4-47b6-b3c6-1b510cc42adc None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:33.204490 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319bcbf-f0f4-47b6-b3c6-1b510cc42adc None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:33.204589 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc8927-ec0d-4223-b85f-e4171018e50d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:33.204765 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319bcbf-f0f4-47b6-b3c6-1b510cc42adc None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:33.204857 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc8927-ec0d-4223-b85f-e4171018e50d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:33.205571 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c319bcbf-f0f4-47b6-b3c6-1b510cc42adc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:33.205887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c319bcbf-f0f4-47b6-b3c6-1b510cc42adc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:33.206275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc8927-ec0d-4223-b85f-e4171018e50d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:33.207910 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c319bcbf-f0f4-47b6-b3c6-1b510cc42adc None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:33.215759 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 551/1093] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:20:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:33.224703 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 543/1094] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:20:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:33.225096 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc8927-ec0d-4223-b85f-e4171018e50d None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:33.225682 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319bcbf-f0f4-47b6-b3c6-1b510cc42adc None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:33.274410 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:33.274571 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:33.325533 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319bcbf-f0f4-47b6-b3c6-1b510cc42adc None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:33.325773 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc8927-ec0d-4223-b85f-e4171018e50d None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:33.358586 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f67228-83e5-4412-a80a-f46333285cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:33.358994 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58faa406-838a-4448-925c-371387f59fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:33.359638 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f67228-83e5-4412-a80a-f46333285cff None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:33.359781 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58faa406-838a-4448-925c-371387f59fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:33.360048 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f67228-83e5-4412-a80a-f46333285cff None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:33.360148 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58faa406-838a-4448-925c-371387f59fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:33.360365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58faa406-838a-4448-925c-371387f59fe4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:33.360468 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f67228-83e5-4412-a80a-f46333285cff None admin] PATH_INFO: `/v3/users/7630f9921f104c57b14fc0bf3e0b20ad` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:33.361606 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f67228-83e5-4412-a80a-f46333285cff None admin] RBAC: Authorizing `identity:get_user(user_id=7630f9921f104c57b14fc0bf3e0b20ad)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:33.363707 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f67228-83e5-4412-a80a-f46333285cff None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:33.365412 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 552/1095] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:20:33 2022] GET /identity/v3/users/7630f9921f104c57b14fc0bf3e0b20ad => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:33.368056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58faa406-838a-4448-925c-371387f59fe4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:33.369526 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58faa406-838a-4448-925c-371387f59fe4 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:33.370965 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f67228-83e5-4412-a80a-f46333285cff None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:33.383894 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-58faa406-838a-4448-925c-371387f59fe4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 12:20:33.383894 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:33.383894 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:33.383894 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:33.383894 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:33.383894 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:33.383894 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:33.383894 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:33.383894 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:33.383894 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:33.383894 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:33.383894 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:33.383894 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:20:33.383894 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:20:33.383894 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:20:33.383894 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:20:33.385129 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:33.385129 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:33.385129 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:20:33.385129 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:20:33.385129 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:20:33.385129 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:20:33.385129 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:33.385129 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:33.385129 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:33.385129 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:33.385129 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:33.385129 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:20:33.385129 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:33.385129 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:33.385129 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:33.385129 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:33.385129 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:33.385129 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:33.386407 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:33.386407 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:33.386407 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:20:33.386407 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:20:33.386407 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:20:33.386407 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:20:33.386407 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:20:33.386407 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:20:33.386407 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 12:20:33.386407 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:33.386407 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 544/1096] 10.222.0.16 () {56 vars in 1093 bytes} [Mon May 23 12:20:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:20:33.390621 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58faa406-838a-4448-925c-371387f59fe4 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:33.473609 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f67228-83e5-4412-a80a-f46333285cff None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:33.489378 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:33.489583 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58faa406-838a-4448-925c-371387f59fe4 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:33.499162 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:33.508486 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d74cd-b184-482e-8f7e-e64791d6535c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:33.509131 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d74cd-b184-482e-8f7e-e64791d6535c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:33.509415 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d74cd-b184-482e-8f7e-e64791d6535c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:33.509694 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d74cd-b184-482e-8f7e-e64791d6535c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:33.525066 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d8855-9284-4d30-aa49-6d8a034754ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:33.525666 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d8855-9284-4d30-aa49-6d8a034754ed None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:33.525948 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d8855-9284-4d30-aa49-6d8a034754ed None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:33.526225 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d8855-9284-4d30-aa49-6d8a034754ed None admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:33.527055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6d8855-9284-4d30-aa49-6d8a034754ed None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:33.527365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6d8855-9284-4d30-aa49-6d8a034754ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:33.528830 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6d8855-9284-4d30-aa49-6d8a034754ed None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:33.546294 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 545/1097] 10.222.0.16 () {56 vars in 1133 bytes} [Mon May 23 12:20:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:33.551874 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d8855-9284-4d30-aa49-6d8a034754ed None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:33.577698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d74cd-b184-482e-8f7e-e64791d6535c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:33.579140 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d74cd-b184-482e-8f7e-e64791d6535c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:33.648724 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d8855-9284-4d30-aa49-6d8a034754ed None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:33.680573 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 553/1098] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:20:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 12:20:33.681904 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d74cd-b184-482e-8f7e-e64791d6535c None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:33.683937 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ff609-e356-4701-84da-0f466bddd2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:33.684610 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ff609-e356-4701-84da-0f466bddd2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:33.684893 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ff609-e356-4701-84da-0f466bddd2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:33.685167 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ff609-e356-4701-84da-0f466bddd2f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:33.699178 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:33.720352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:33.755089 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7ff609-e356-4701-84da-0f466bddd2f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:33.756571 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7ff609-e356-4701-84da-0f466bddd2f2 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:33.781396 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d74cd-b184-482e-8f7e-e64791d6535c None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:33.815167 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687fc37-daf2-4618-9ab7-ea82f15b78ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:33.816049 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687fc37-daf2-4618-9ab7-ea82f15b78ce None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:33.816476 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687fc37-daf2-4618-9ab7-ea82f15b78ce None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:33.816776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d687fc37-daf2-4618-9ab7-ea82f15b78ce None admin] PATH_INFO: `/v3/projects/c6d0602ebd9c4e1083e8d05e49f38548` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:33.818318 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687fc37-daf2-4618-9ab7-ea82f15b78ce None admin] RBAC: Authorizing `identity:get_project(project_id=c6d0602ebd9c4e1083e8d05e49f38548)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:33.819738 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d687fc37-daf2-4618-9ab7-ea82f15b78ce None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:33.821464 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 554/1099] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:20:33 2022] GET /identity/v3/projects/c6d0602ebd9c4e1083e8d05e49f38548 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:33.828150 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687fc37-daf2-4618-9ab7-ea82f15b78ce None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:33.859555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 546/1100] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:20:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 12:20:33.865590 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ff609-e356-4701-84da-0f466bddd2f2 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:33.914835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:33.930026 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:33.930385 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d687fc37-daf2-4618-9ab7-ea82f15b78ce None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:33.965307 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ff609-e356-4701-84da-0f466bddd2f2 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:33.966637 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafb2da-fac6-4e1a-9add-7fedcff9b26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:33.967284 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafb2da-fac6-4e1a-9add-7fedcff9b26e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:33.967566 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafb2da-fac6-4e1a-9add-7fedcff9b26e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:33.967894 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafb2da-fac6-4e1a-9add-7fedcff9b26e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:33.968772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aafb2da-fac6-4e1a-9add-7fedcff9b26e None admin] RBAC: Adding query filter params (role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8, scope.project.id=c6d0602ebd9c4e1083e8d05e49f38548, user.id=7630f9921f104c57b14fc0bf3e0b20ad) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:33.969081 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aafb2da-fac6-4e1a-9add-7fedcff9b26e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:33.970489 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aafb2da-fac6-4e1a-9add-7fedcff9b26e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:33.976384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 555/1101] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:20:33 2022] GET /identity/v3/role_assignments?user.id=7630f9921f104c57b14fc0bf3e0b20ad&scope.project.id=c6d0602ebd9c4e1083e8d05e49f38548&role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:34.003157 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93861a-570b-4659-9012-40da89886e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:34.004920 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93861a-570b-4659-9012-40da89886e4c None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:34.004920 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93861a-570b-4659-9012-40da89886e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:34.005141 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93861a-570b-4659-9012-40da89886e4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:34.016369 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a93861a-570b-4659-9012-40da89886e4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:34.017867 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a93861a-570b-4659-9012-40da89886e4c None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:34.037612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-1a93861a-570b-4659-9012-40da89886e4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:20:34.037612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:34.037612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:34.037612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:34.037612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:34.037612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:34.037612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:34.037612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:34.037612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:34.037612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:34.037612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:34.037612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:34.037612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:20:34.037612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:20:34.037612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:20:34.038750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:20:34.038750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:34.038750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:34.038750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:34.038750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:34.038750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:34.038750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:34.038750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:34.038750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:20:34.038750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:34.038750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:34.038750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:34.038750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:34.038750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:34.038750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:34.038750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:34.038750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:34.039714 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:20:34.039714 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:20:34.039714 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:20:34.039714 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:34.039714 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 547/1102] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:20:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:34.044792 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafb2da-fac6-4e1a-9add-7fedcff9b26e None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:34.153672 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafb2da-fac6-4e1a-9add-7fedcff9b26e None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:34.191649 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aae0da-12e2-40d4-9b37-4cbca5db510c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:34.192840 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aae0da-12e2-40d4-9b37-4cbca5db510c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:34.193129 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aae0da-12e2-40d4-9b37-4cbca5db510c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:34.193408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aae0da-12e2-40d4-9b37-4cbca5db510c None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:34.194133 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aae0da-12e2-40d4-9b37-4cbca5db510c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:34.194440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aae0da-12e2-40d4-9b37-4cbca5db510c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:34.195802 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aae0da-12e2-40d4-9b37-4cbca5db510c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:34.208326 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 556/1103] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:34.214108 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93861a-570b-4659-9012-40da89886e4c None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:34.312702 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93861a-570b-4659-9012-40da89886e4c None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:34.330318 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:34.348045 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8e685-44cb-4d5c-af59-352f69037483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:34.348990 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8e685-44cb-4d5c-af59-352f69037483 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:34.349275 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8e685-44cb-4d5c-af59-352f69037483 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:34.349551 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8e685-44cb-4d5c-af59-352f69037483 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:34.356076 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b8e685-44cb-4d5c-af59-352f69037483 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:34.357541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b8e685-44cb-4d5c-af59-352f69037483 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:34.371578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-84b8e685-44cb-4d5c-af59-352f69037483 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:20:34.371578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:34.371578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:34.371578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:34.371578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:34.371578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:34.371578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:34.371578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:34.371578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:34.371578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:34.371578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:34.371578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:34.371578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:20:34.371578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:20:34.371578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:20:34.372606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:20:34.372606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:34.372606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:34.372606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:34.372606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:34.372606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:34.372606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:34.372606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:34.372606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:20:34.372606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:34.372606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:34.372606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:34.372606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:34.372606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:34.372606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:34.372606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:34.372606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:34.373516 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:20:34.373516 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:20:34.373516 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:20:34.373516 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:20:34.373516 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:20:34.373516 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:20:34.373516 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:20:34.373516 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:20:34.373516 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 548/1104] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:20:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:34.378714 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aae0da-12e2-40d4-9b37-4cbca5db510c None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:34.389716 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:34.486499 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aae0da-12e2-40d4-9b37-4cbca5db510c None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:34.521847 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c72be-f2b3-4fca-bdb4-177577426730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:34.522508 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c72be-f2b3-4fca-bdb4-177577426730 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:34.522790 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c72be-f2b3-4fca-bdb4-177577426730 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:34.523067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c72be-f2b3-4fca-bdb4-177577426730 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:34.523849 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c72be-f2b3-4fca-bdb4-177577426730 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:34.524162 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c72be-f2b3-4fca-bdb4-177577426730 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:34.525612 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c72be-f2b3-4fca-bdb4-177577426730 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:34.537832 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 557/1105] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:20:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:20:34.543835 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8e685-44cb-4d5c-af59-352f69037483 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:34.641545 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8e685-44cb-4d5c-af59-352f69037483 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:34.674104 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5790f-3e72-4bc8-a177-edf2b29303cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:34.674754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5790f-3e72-4bc8-a177-edf2b29303cb None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:34.675037 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5790f-3e72-4bc8-a177-edf2b29303cb None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:34.675311 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5790f-3e72-4bc8-a177-edf2b29303cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:34.676666 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5790f-3e72-4bc8-a177-edf2b29303cb None admin] RBAC: Adding query filter params (role.id=3d40411f939f4ff6af2e78c756f6cc9e, scope.project.id=839585501f414e1ab02501422510e54f, user.id=b83535b443a84275a160cd84cb3ad5a0) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:34.676975 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5790f-3e72-4bc8-a177-edf2b29303cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:34.678404 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5790f-3e72-4bc8-a177-edf2b29303cb None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:34.684221 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 549/1106] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:20:34 2022] GET /identity/v3/role_assignments?user.id=b83535b443a84275a160cd84cb3ad5a0&scope.project.id=839585501f414e1ab02501422510e54f&role.id=3d40411f939f4ff6af2e78c756f6cc9e => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:36.238737 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bd84a-30b6-4c9a-b4d4-ba0017f6a970 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:36.239269 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bd84a-30b6-4c9a-b4d4-ba0017f6a970 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:36.239269 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bd84a-30b6-4c9a-b4d4-ba0017f6a970 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:36.240666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 558/1107] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:36.246525 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5c7b7-f309-4267-9b3f-64365f44de76 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:36.246960 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5c7b7-f309-4267-9b3f-64365f44de76 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:36.247088 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5c7b7-f309-4267-9b3f-64365f44de76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:36.263624 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-f8d5c7b7-f309-4267-9b3f-64365f44de76 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:36.264831 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-f8d5c7b7-f309-4267-9b3f-64365f44de76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:36.298670 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:36.333890 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 550/1108] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:20:36.358708 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3bd84a-30b6-4c9a-b4d4-ba0017f6a970 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:36.424852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:36.459163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3bd84a-30b6-4c9a-b4d4-ba0017f6a970 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:36.492150 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74cff80-4d92-4d1d-972c-b751a601b9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:36.492958 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cff80-4d92-4d1d-972c-b751a601b9fc None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:36.493240 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cff80-4d92-4d1d-972c-b751a601b9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:36.493516 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74cff80-4d92-4d1d-972c-b751a601b9fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:36.494676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74cff80-4d92-4d1d-972c-b751a601b9fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:36.496094 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74cff80-4d92-4d1d-972c-b751a601b9fc None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:36.497700 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 559/1109] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:20:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:36.503407 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5c7b7-f309-4267-9b3f-64365f44de76 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:36.599979 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5c7b7-f309-4267-9b3f-64365f44de76 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:36.632240 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ef8e0-a213-42fd-9627-6612466162f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:36.632901 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ef8e0-a213-42fd-9627-6612466162f0 None admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:36.633186 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ef8e0-a213-42fd-9627-6612466162f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:36.633465 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ef8e0-a213-42fd-9627-6612466162f0 None admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:36.634172 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7ef8e0-a213-42fd-9627-6612466162f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:36.635607 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7ef8e0-a213-42fd-9627-6612466162f0 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:36.655225 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 551/1110] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 12:20:36 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:20:38.884528 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5fbbf-8991-432f-bceb-fe63274b903c None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:38.885101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5fbbf-8991-432f-bceb-fe63274b903c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:38.885101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5fbbf-8991-432f-bceb-fe63274b903c None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:38.886454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 560/1111] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:38.891609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e67037-799f-47d5-9dc0-eb04cf7cec8a None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:38.891999 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e67037-799f-47d5-9dc0-eb04cf7cec8a None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:38.892190 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e67037-799f-47d5-9dc0-eb04cf7cec8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:38.909038 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-08e67037-799f-47d5-9dc0-eb04cf7cec8a None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:38.910839 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-08e67037-799f-47d5-9dc0-eb04cf7cec8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:38.932790 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:38.977671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 552/1112] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:20:39.000950 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b5fbbf-8991-432f-bceb-fe63274b903c None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:39.099144 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b5fbbf-8991-432f-bceb-fe63274b903c None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:39.130725 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d90965-1da4-4a11-9169-792cf68028f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:39.131565 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d90965-1da4-4a11-9169-792cf68028f9 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:39.131846 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d90965-1da4-4a11-9169-792cf68028f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:39.132119 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d90965-1da4-4a11-9169-792cf68028f9 None admin] PATH_INFO: `/v3/roles/2d4732dacad6444e86d7fee3514afefb` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:39.133578 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d90965-1da4-4a11-9169-792cf68028f9 None admin] RBAC: Authorizing `identity:get_role(role_id=2d4732dacad6444e86d7fee3514afefb)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:39.134969 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d90965-1da4-4a11-9169-792cf68028f9 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:39.136285 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 561/1113] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:20:38 2022] GET /identity/v3/roles/2d4732dacad6444e86d7fee3514afefb => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:39.141944 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e67037-799f-47d5-9dc0-eb04cf7cec8a None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:39.237582 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e67037-799f-47d5-9dc0-eb04cf7cec8a None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:39.270294 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f061f-2853-4930-8fc7-1a3c51124136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:39.271137 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f061f-2853-4930-8fc7-1a3c51124136 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:39.271425 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f061f-2853-4930-8fc7-1a3c51124136 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:39.271705 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f061f-2853-4930-8fc7-1a3c51124136 None admin] PATH_INFO: `/v3/users/83104aeb9dda460fb63df6a7dfe1db6f` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:39.284252 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f061f-2853-4930-8fc7-1a3c51124136 None admin] RBAC: Authorizing `identity:get_user(user_id=83104aeb9dda460fb63df6a7dfe1db6f)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:39.286341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f061f-2853-4930-8fc7-1a3c51124136 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:39.287941 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 553/1114] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:20:39 2022] GET /identity/v3/users/83104aeb9dda460fb63df6a7dfe1db6f => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:39.293549 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d90965-1da4-4a11-9169-792cf68028f9 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:39.297223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:39.390098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d90965-1da4-4a11-9169-792cf68028f9 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:39.422856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc17cb-1b6a-49a4-b21e-272ef2acc964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:39.423495 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc17cb-1b6a-49a4-b21e-272ef2acc964 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:39.423783 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc17cb-1b6a-49a4-b21e-272ef2acc964 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:39.424066 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc17cb-1b6a-49a4-b21e-272ef2acc964 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:39.491539 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc17cb-1b6a-49a4-b21e-272ef2acc964 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:39.493022 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc17cb-1b6a-49a4-b21e-272ef2acc964 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:39.495568 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:39.588183 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 562/1115] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:20:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 12:20:39.594016 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f061f-2853-4930-8fc7-1a3c51124136 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:39.612133 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:39.691590 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f061f-2853-4930-8fc7-1a3c51124136 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:39.725014 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476ade1-03aa-4575-99b9-4edefb958228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:39.725848 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476ade1-03aa-4575-99b9-4edefb958228 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:39.726135 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476ade1-03aa-4575-99b9-4edefb958228 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:39.726413 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476ade1-03aa-4575-99b9-4edefb958228 None admin] PATH_INFO: `/v3/projects/c6d0602ebd9c4e1083e8d05e49f38548` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:39.727570 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476ade1-03aa-4575-99b9-4edefb958228 None admin] RBAC: Authorizing `identity:get_project(project_id=c6d0602ebd9c4e1083e8d05e49f38548)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:39.729032 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476ade1-03aa-4575-99b9-4edefb958228 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:39.730679 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 554/1116] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:20:39 2022] GET /identity/v3/projects/c6d0602ebd9c4e1083e8d05e49f38548 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:39.736632 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc17cb-1b6a-49a4-b21e-272ef2acc964 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:39.851322 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc17cb-1b6a-49a4-b21e-272ef2acc964 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:39.853594 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:39.885063 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798ab5a7-0178-46ec-a24f-158d39f0af2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:39.885711 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ab5a7-0178-46ec-a24f-158d39f0af2a None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:39.885988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ab5a7-0178-46ec-a24f-158d39f0af2a None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:39.886261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798ab5a7-0178-46ec-a24f-158d39f0af2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:39.887076 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798ab5a7-0178-46ec-a24f-158d39f0af2a None admin] RBAC: Adding query filter params (role.id=2d4732dacad6444e86d7fee3514afefb, scope.project.id=c6d0602ebd9c4e1083e8d05e49f38548, user.id=83104aeb9dda460fb63df6a7dfe1db6f) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:39.887384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798ab5a7-0178-46ec-a24f-158d39f0af2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:39.888830 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798ab5a7-0178-46ec-a24f-158d39f0af2a None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:39.894843 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 563/1117] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:20:39 2022] GET /identity/v3/role_assignments?user.id=83104aeb9dda460fb63df6a7dfe1db6f&scope.project.id=c6d0602ebd9c4e1083e8d05e49f38548&role.id=2d4732dacad6444e86d7fee3514afefb => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:42.061838 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391e17e-b049-44e7-8341-9b3437e21a90 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:42.062291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391e17e-b049-44e7-8341-9b3437e21a90 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:42.062404 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391e17e-b049-44e7-8341-9b3437e21a90 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:42.063734 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 555/1118] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:42.069184 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6556836-6ad3-43de-b354-4440d55e408c None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:42.069572 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6556836-6ad3-43de-b354-4440d55e408c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:42.069747 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6556836-6ad3-43de-b354-4440d55e408c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:42.086681 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-b6556836-6ad3-43de-b354-4440d55e408c None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:42.087865 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b6556836-6ad3-43de-b354-4440d55e408c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:42.152032 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 564/1119] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:20:42.174693 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5391e17e-b049-44e7-8341-9b3437e21a90 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:42.262771 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:42.278888 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5391e17e-b049-44e7-8341-9b3437e21a90 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:42.312270 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0659cbb-b29e-42ea-aea6-1809c811c6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:42.313113 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0659cbb-b29e-42ea-aea6-1809c811c6a3 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:42.313395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0659cbb-b29e-42ea-aea6-1809c811c6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:42.313670 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0659cbb-b29e-42ea-aea6-1809c811c6a3 None admin] PATH_INFO: `/v3/roles/2d4732dacad6444e86d7fee3514afefb` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:42.315138 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0659cbb-b29e-42ea-aea6-1809c811c6a3 None admin] RBAC: Authorizing `identity:get_role(role_id=2d4732dacad6444e86d7fee3514afefb)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:42.316744 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0659cbb-b29e-42ea-aea6-1809c811c6a3 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:42.318348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 556/1120] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:20:42 2022] GET /identity/v3/roles/2d4732dacad6444e86d7fee3514afefb => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:42.324095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6556836-6ad3-43de-b354-4440d55e408c None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:42.416965 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:42.420825 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6556836-6ad3-43de-b354-4440d55e408c None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:42.454288 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55b1a1-64cc-4052-913c-acd884b35941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:42.455144 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55b1a1-64cc-4052-913c-acd884b35941 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:42.455425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55b1a1-64cc-4052-913c-acd884b35941 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:42.455696 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55b1a1-64cc-4052-913c-acd884b35941 None admin] PATH_INFO: `/v3/users/83104aeb9dda460fb63df6a7dfe1db6f` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:42.457225 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce55b1a1-64cc-4052-913c-acd884b35941 None admin] RBAC: Authorizing `identity:get_user(user_id=83104aeb9dda460fb63df6a7dfe1db6f)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:42.458640 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce55b1a1-64cc-4052-913c-acd884b35941 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:42.460243 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 565/1121] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:20:42 2022] GET /identity/v3/users/83104aeb9dda460fb63df6a7dfe1db6f => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:42.466198 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0659cbb-b29e-42ea-aea6-1809c811c6a3 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:42.562549 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0659cbb-b29e-42ea-aea6-1809c811c6a3 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:42.596654 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d07d2-c3e1-4481-92a5-8dc9d810897b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:42.598061 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d07d2-c3e1-4481-92a5-8dc9d810897b None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:42.598390 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d07d2-c3e1-4481-92a5-8dc9d810897b None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:42.598665 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d07d2-c3e1-4481-92a5-8dc9d810897b None admin] PATH_INFO: `/v3/projects/c6d0602ebd9c4e1083e8d05e49f38548` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:42.599829 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d07d2-c3e1-4481-92a5-8dc9d810897b None admin] RBAC: Authorizing `identity:get_project(project_id=c6d0602ebd9c4e1083e8d05e49f38548)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:42.601286 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d07d2-c3e1-4481-92a5-8dc9d810897b None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:42.602921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 557/1122] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:20:42 2022] GET /identity/v3/projects/c6d0602ebd9c4e1083e8d05e49f38548 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:42.609024 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55b1a1-64cc-4052-913c-acd884b35941 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:42.704992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55b1a1-64cc-4052-913c-acd884b35941 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:42.737986 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936705d7-945b-4b11-a03a-eaf79fcf7730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:42.738859 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936705d7-945b-4b11-a03a-eaf79fcf7730 None admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:42.739133 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936705d7-945b-4b11-a03a-eaf79fcf7730 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:42.739404 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936705d7-945b-4b11-a03a-eaf79fcf7730 None admin] PATH_INFO: `/v3/projects/c6d0602ebd9c4e1083e8d05e49f38548/users/83104aeb9dda460fb63df6a7dfe1db6f/roles/2d4732dacad6444e86d7fee3514afefb` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:42.741571 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936705d7-945b-4b11-a03a-eaf79fcf7730 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6d0602ebd9c4e1083e8d05e49f38548, user_id=83104aeb9dda460fb63df6a7dfe1db6f, role_id=2d4732dacad6444e86d7fee3514afefb)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:42.742968 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936705d7-945b-4b11-a03a-eaf79fcf7730 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:42.755150 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 566/1123] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 12:20:42 2022] PUT /identity/v3/projects/c6d0602ebd9c4e1083e8d05e49f38548/users/83104aeb9dda460fb63df6a7dfe1db6f/roles/2d4732dacad6444e86d7fee3514afefb => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:20:44.944410 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c45d0d-7a8f-4bfa-874b-c4b430bfe990 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:44.944943 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c45d0d-7a8f-4bfa-874b-c4b430bfe990 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:44.944943 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c45d0d-7a8f-4bfa-874b-c4b430bfe990 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:44.946296 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 558/1124] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:44.951429 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec66fcdf-b135-4bfc-a5e7-e97c2f0d23cd None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:44.951823 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec66fcdf-b135-4bfc-a5e7-e97c2f0d23cd None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:44.951981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec66fcdf-b135-4bfc-a5e7-e97c2f0d23cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:44.968922 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-ec66fcdf-b135-4bfc-a5e7-e97c2f0d23cd None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:44.970081 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ec66fcdf-b135-4bfc-a5e7-e97c2f0d23cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:44.990593 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:45.037665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 567/1125] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:20:45.060706 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c45d0d-7a8f-4bfa-874b-c4b430bfe990 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:45.082671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:45.158916 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c45d0d-7a8f-4bfa-874b-c4b430bfe990 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:45.192051 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef07b9-9298-4fc7-9bfb-c5464ef69374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:45.193173 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef07b9-9298-4fc7-9bfb-c5464ef69374 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:45.193459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef07b9-9298-4fc7-9bfb-c5464ef69374 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:45.193736 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef07b9-9298-4fc7-9bfb-c5464ef69374 None admin] PATH_INFO: `/v3/roles/2d4732dacad6444e86d7fee3514afefb` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:45.195243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef07b9-9298-4fc7-9bfb-c5464ef69374 None admin] RBAC: Authorizing `identity:get_role(role_id=2d4732dacad6444e86d7fee3514afefb)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:45.196607 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef07b9-9298-4fc7-9bfb-c5464ef69374 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:45.197935 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 559/1126] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:20:45 2022] GET /identity/v3/roles/2d4732dacad6444e86d7fee3514afefb => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:45.203632 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec66fcdf-b135-4bfc-a5e7-e97c2f0d23cd None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:45.299685 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec66fcdf-b135-4bfc-a5e7-e97c2f0d23cd None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:45.334409 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c2645f-d9db-4e8f-b43b-c590f671b072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:45.335284 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2645f-d9db-4e8f-b43b-c590f671b072 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:45.335566 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2645f-d9db-4e8f-b43b-c590f671b072 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:45.335844 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2645f-d9db-4e8f-b43b-c590f671b072 None admin] PATH_INFO: `/v3/users/83104aeb9dda460fb63df6a7dfe1db6f` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:45.337496 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c2645f-d9db-4e8f-b43b-c590f671b072 None admin] RBAC: Authorizing `identity:get_user(user_id=83104aeb9dda460fb63df6a7dfe1db6f)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:45.338472 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c2645f-d9db-4e8f-b43b-c590f671b072 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:45.340083 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 568/1127] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:20:45 2022] GET /identity/v3/users/83104aeb9dda460fb63df6a7dfe1db6f => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:45.345843 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef07b9-9298-4fc7-9bfb-c5464ef69374 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:45.438932 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:45.451724 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef07b9-9298-4fc7-9bfb-c5464ef69374 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:45.484430 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87978f5f-9404-4070-9365-9cd732bef321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:45.485069 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87978f5f-9404-4070-9365-9cd732bef321 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:45.485356 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87978f5f-9404-4070-9365-9cd732bef321 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:45.485635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87978f5f-9404-4070-9365-9cd732bef321 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:45.549688 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87978f5f-9404-4070-9365-9cd732bef321 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:45.551367 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87978f5f-9404-4070-9365-9cd732bef321 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:45.634117 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:45.646637 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 560/1128] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:20:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 12:20:45.652690 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c2645f-d9db-4e8f-b43b-c590f671b072 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:45.677932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:45.751059 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c2645f-d9db-4e8f-b43b-c590f671b072 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:45.784704 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7a3400-6bab-442a-b1b6-303d7958b572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:45.785647 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7a3400-6bab-442a-b1b6-303d7958b572 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:45.785931 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7a3400-6bab-442a-b1b6-303d7958b572 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:45.786209 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7a3400-6bab-442a-b1b6-303d7958b572 None admin] PATH_INFO: `/v3/projects/c6d0602ebd9c4e1083e8d05e49f38548` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:45.787370 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7a3400-6bab-442a-b1b6-303d7958b572 None admin] RBAC: Authorizing `identity:get_project(project_id=c6d0602ebd9c4e1083e8d05e49f38548)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:45.788844 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7a3400-6bab-442a-b1b6-303d7958b572 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:45.790511 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 569/1129] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:20:45 2022] GET /identity/v3/projects/c6d0602ebd9c4e1083e8d05e49f38548 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:45.796576 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87978f5f-9404-4070-9365-9cd732bef321 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:45.894005 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87978f5f-9404-4070-9365-9cd732bef321 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:45.929399 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2e6b4-2479-478d-a521-2f73b011810d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:45.930056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2e6b4-2479-478d-a521-2f73b011810d None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:45.930344 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2e6b4-2479-478d-a521-2f73b011810d None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:45.930624 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2e6b4-2479-478d-a521-2f73b011810d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:45.931480 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2e6b4-2479-478d-a521-2f73b011810d None admin] RBAC: Adding query filter params (role.id=2d4732dacad6444e86d7fee3514afefb, scope.project.id=c6d0602ebd9c4e1083e8d05e49f38548, user.id=83104aeb9dda460fb63df6a7dfe1db6f) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:45.931795 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2e6b4-2479-478d-a521-2f73b011810d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:45.933269 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2e6b4-2479-478d-a521-2f73b011810d None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:45.939124 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 561/1130] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:20:45 2022] GET /identity/v3/role_assignments?user.id=83104aeb9dda460fb63df6a7dfe1db6f&scope.project.id=c6d0602ebd9c4e1083e8d05e49f38548&role.id=2d4732dacad6444e86d7fee3514afefb => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:48.137716 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cbf0e3-38f9-4b38-89d1-19d8a316cec6 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:48.138262 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cbf0e3-38f9-4b38-89d1-19d8a316cec6 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:48.138262 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cbf0e3-38f9-4b38-89d1-19d8a316cec6 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:48.139616 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 570/1131] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:48.144800 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69ede1-73dc-4b19-9a21-fe97fa8cd7dd None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:48.145214 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69ede1-73dc-4b19-9a21-fe97fa8cd7dd None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:48.145362 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d69ede1-73dc-4b19-9a21-fe97fa8cd7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:48.161785 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-3d69ede1-73dc-4b19-9a21-fe97fa8cd7dd None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:48.162962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-3d69ede1-73dc-4b19-9a21-fe97fa8cd7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:48.209034 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:48.232090 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 562/1132] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:20:48.255691 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cbf0e3-38f9-4b38-89d1-19d8a316cec6 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:48.345945 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:48.356424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cbf0e3-38f9-4b38-89d1-19d8a316cec6 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:48.388959 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb488641-d644-437c-8c3a-12b849ea7420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:48.389732 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb488641-d644-437c-8c3a-12b849ea7420 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:48.390012 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb488641-d644-437c-8c3a-12b849ea7420 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:48.390286 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb488641-d644-437c-8c3a-12b849ea7420 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:48.391426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb488641-d644-437c-8c3a-12b849ea7420 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:48.392869 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb488641-d644-437c-8c3a-12b849ea7420 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:48.394551 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 571/1133] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:20:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:48.400369 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69ede1-73dc-4b19-9a21-fe97fa8cd7dd None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:48.497176 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d69ede1-73dc-4b19-9a21-fe97fa8cd7dd None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:48.529225 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99879135-b493-4767-a5d6-a06049d5f4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:48.529872 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99879135-b493-4767-a5d6-a06049d5f4ba None admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:48.530156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99879135-b493-4767-a5d6-a06049d5f4ba None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:48.530432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99879135-b493-4767-a5d6-a06049d5f4ba None admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:48.531136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99879135-b493-4767-a5d6-a06049d5f4ba None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:48.532599 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99879135-b493-4767-a5d6-a06049d5f4ba None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:48.557492 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 563/1134] 10.222.0.16 () {60 vars in 1141 bytes} [Mon May 23 12:20:48 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:20:50.803514 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2861746-ee29-4a9f-8ea3-9cf15dc9ec1c None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:50.804067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2861746-ee29-4a9f-8ea3-9cf15dc9ec1c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:50.804067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2861746-ee29-4a9f-8ea3-9cf15dc9ec1c None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:50.805464 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 572/1135] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:50.810939 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab6ed4-a26d-4ba2-b81c-7900d27366ac None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:50.811354 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab6ed4-a26d-4ba2-b81c-7900d27366ac None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:50.811505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab6ed4-a26d-4ba2-b81c-7900d27366ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:50.828747 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-4aab6ed4-a26d-4ba2-b81c-7900d27366ac None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:50.829920 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-4aab6ed4-a26d-4ba2-b81c-7900d27366ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:50.856011 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:50.897741 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 564/1136] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:20:50.922299 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2861746-ee29-4a9f-8ea3-9cf15dc9ec1c None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:51.023618 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2861746-ee29-4a9f-8ea3-9cf15dc9ec1c None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:51.058587 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6f0e9-8bd2-413c-8261-912f6fafd5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:51.059306 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6f0e9-8bd2-413c-8261-912f6fafd5a9 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:51.059600 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6f0e9-8bd2-413c-8261-912f6fafd5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:51.059876 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6f0e9-8bd2-413c-8261-912f6fafd5a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:51.061070 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be6f0e9-8bd2-413c-8261-912f6fafd5a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:51.062493 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be6f0e9-8bd2-413c-8261-912f6fafd5a9 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:51.064094 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 573/1137] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:20:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:51.070986 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab6ed4-a26d-4ba2-b81c-7900d27366ac None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:51.166576 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab6ed4-a26d-4ba2-b81c-7900d27366ac None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:51.201072 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e35f09-00d5-491a-9a5b-39d7afb2ddf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:51.201727 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e35f09-00d5-491a-9a5b-39d7afb2ddf2 None admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:51.202008 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e35f09-00d5-491a-9a5b-39d7afb2ddf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:51.202282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e35f09-00d5-491a-9a5b-39d7afb2ddf2 None admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:51.202990 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e35f09-00d5-491a-9a5b-39d7afb2ddf2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:51.204445 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e35f09-00d5-491a-9a5b-39d7afb2ddf2 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:51.223941 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 565/1138] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 12:20:51 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:20:53.462939 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f36fbc-ccef-4e42-b31c-adcdeea96745 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:53.463469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f36fbc-ccef-4e42-b31c-adcdeea96745 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:53.463469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f36fbc-ccef-4e42-b31c-adcdeea96745 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:53.464893 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 574/1139] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:53.470176 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c73cb-915b-4423-9615-9dea6d926188 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:53.470603 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c73cb-915b-4423-9615-9dea6d926188 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:53.470734 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c73cb-915b-4423-9615-9dea6d926188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:53.488244 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-652c73cb-915b-4423-9615-9dea6d926188 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:53.489422 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-652c73cb-915b-4423-9615-9dea6d926188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:53.502861 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:53.554356 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 566/1140] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:20:53.577768 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f36fbc-ccef-4e42-b31c-adcdeea96745 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:53.605550 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:53.677472 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f36fbc-ccef-4e42-b31c-adcdeea96745 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:53.710289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21324998-18e2-4096-9346-e6837359f3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:53.711066 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21324998-18e2-4096-9346-e6837359f3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:53.711350 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21324998-18e2-4096-9346-e6837359f3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:53.711627 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21324998-18e2-4096-9346-e6837359f3e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:53.721542 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21324998-18e2-4096-9346-e6837359f3e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:20:53.729704 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-21324998-18e2-4096-9346-e6837359f3e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:20:53.729704 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:53.729704 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:53.729704 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:53.729704 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:53.729704 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:53.729704 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:53.729704 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:53.729704 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:53.729704 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:53.729704 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:53.729704 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:53.729704 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:20:53.729704 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:20:53.729704 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:20:53.729704 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:20:53.730801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:20:53.730801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:20:53.730801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:20:53.730801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:20:53.730801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:53.730801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:53.730801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:53.730801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:53.730801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:53.730801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:53.730801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:53.730801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:20:53.730801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:53.730801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:53.730801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:53.730801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:53.731719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:53.731719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:53.731719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:53.731719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:53.731719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:20:53.731719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:20:53.731719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:20:53.731719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:20:53.731719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:20:53.731719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:20:53.731719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:20:53.731719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:53.731719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 575/1141] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:20:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:53.736938 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c73cb-915b-4423-9615-9dea6d926188 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:53.833542 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c73cb-915b-4423-9615-9dea6d926188 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:53.866677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837eaaf4-0210-44b4-9679-f198e21b9e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:53.867327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837eaaf4-0210-44b4-9679-f198e21b9e1e None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:53.867609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837eaaf4-0210-44b4-9679-f198e21b9e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:53.867884 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837eaaf4-0210-44b4-9679-f198e21b9e1e None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:53.868650 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837eaaf4-0210-44b4-9679-f198e21b9e1e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:53.868957 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837eaaf4-0210-44b4-9679-f198e21b9e1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:53.870317 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837eaaf4-0210-44b4-9679-f198e21b9e1e None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:53.878211 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 567/1142] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:20:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:53.883815 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21324998-18e2-4096-9346-e6837359f3e9 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:53.969601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:53.982136 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21324998-18e2-4096-9346-e6837359f3e9 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:54.015242 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5944f-fabb-4572-bf7c-f012ca07f1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:54.016115 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5944f-fabb-4572-bf7c-f012ca07f1df None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:54.016433 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5944f-fabb-4572-bf7c-f012ca07f1df None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:54.016717 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5944f-fabb-4572-bf7c-f012ca07f1df None admin] PATH_INFO: `/v3/users/0836af286ae743f199e263416caa1daa` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:54.030824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5944f-fabb-4572-bf7c-f012ca07f1df None admin] RBAC: Authorizing `identity:get_user(user_id=0836af286ae743f199e263416caa1daa)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:54.032297 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5944f-fabb-4572-bf7c-f012ca07f1df None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:54.033984 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 576/1143] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:20:53 2022] GET /identity/v3/users/0836af286ae743f199e263416caa1daa => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:54.040463 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837eaaf4-0210-44b4-9679-f198e21b9e1e None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:54.055555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:54.136979 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837eaaf4-0210-44b4-9679-f198e21b9e1e None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:54.169438 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba50e25-98f1-4cbf-a5b9-533f2f315b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:54.170071 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba50e25-98f1-4cbf-a5b9-533f2f315b1f None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:54.170354 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba50e25-98f1-4cbf-a5b9-533f2f315b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:54.170630 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba50e25-98f1-4cbf-a5b9-533f2f315b1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:54.235994 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba50e25-98f1-4cbf-a5b9-533f2f315b1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:54.237516 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba50e25-98f1-4cbf-a5b9-533f2f315b1f None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:54.250169 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:54.337907 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 568/1144] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:20:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 12:20:54.343757 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5944f-fabb-4572-bf7c-f012ca07f1df None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:54.439847 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5944f-fabb-4572-bf7c-f012ca07f1df None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:54.472039 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b7358-208a-42e8-8735-41e5b1f663da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:54.473419 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b7358-208a-42e8-8735-41e5b1f663da None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:54.473697 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b7358-208a-42e8-8735-41e5b1f663da None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:54.473971 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b7358-208a-42e8-8735-41e5b1f663da None admin] PATH_INFO: `/v3/projects/a974f54b640140a2b5242a47516196b1` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:54.475190 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b7358-208a-42e8-8735-41e5b1f663da None admin] RBAC: Authorizing `identity:get_project(project_id=a974f54b640140a2b5242a47516196b1)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:54.476646 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b7358-208a-42e8-8735-41e5b1f663da None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:54.478370 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 577/1145] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:20:54 2022] GET /identity/v3/projects/a974f54b640140a2b5242a47516196b1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:54.484383 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba50e25-98f1-4cbf-a5b9-533f2f315b1f None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:54.580474 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba50e25-98f1-4cbf-a5b9-533f2f315b1f None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:54.593512 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:54.614210 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a2a05-c1c4-4a52-9660-19d7bfcc7b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:54.614844 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a2a05-c1c4-4a52-9660-19d7bfcc7b4d None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:54.615123 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a2a05-c1c4-4a52-9660-19d7bfcc7b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:54.615394 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a2a05-c1c4-4a52-9660-19d7bfcc7b4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:54.616228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6a2a05-c1c4-4a52-9660-19d7bfcc7b4d None admin] RBAC: Adding query filter params (role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8, scope.project.id=a974f54b640140a2b5242a47516196b1, user.id=0836af286ae743f199e263416caa1daa) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:54.616540 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6a2a05-c1c4-4a52-9660-19d7bfcc7b4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:54.617958 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6a2a05-c1c4-4a52-9660-19d7bfcc7b4d None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:54.623175 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 569/1146] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:20:54 2022] GET /identity/v3/role_assignments?user.id=0836af286ae743f199e263416caa1daa&scope.project.id=a974f54b640140a2b5242a47516196b1&role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:56.818362 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771cfda0-bc35-4789-b2ee-4f8d2d78d924 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:56.818934 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771cfda0-bc35-4789-b2ee-4f8d2d78d924 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:56.818934 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771cfda0-bc35-4789-b2ee-4f8d2d78d924 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:56.820341 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 578/1147] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:56.825506 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46592a5f-98ff-4943-813c-9dffdd587e4b None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:56.825899 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46592a5f-98ff-4943-813c-9dffdd587e4b None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:56.826057 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46592a5f-98ff-4943-813c-9dffdd587e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:56.842794 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-46592a5f-98ff-4943-813c-9dffdd587e4b None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:56.843957 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-46592a5f-98ff-4943-813c-9dffdd587e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:56.908941 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 570/1148] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:20:56.931974 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771cfda0-bc35-4789-b2ee-4f8d2d78d924 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:56.947591 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:57.033933 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771cfda0-bc35-4789-b2ee-4f8d2d78d924 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:57.068329 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf2cb85-dd17-4eb5-a5c1-a938840b8ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:57.069173 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf2cb85-dd17-4eb5-a5c1-a938840b8ecf None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:57.069403 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf2cb85-dd17-4eb5-a5c1-a938840b8ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:57.069674 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf2cb85-dd17-4eb5-a5c1-a938840b8ecf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:57.079478 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf2cb85-dd17-4eb5-a5c1-a938840b8ecf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:20:57.087561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-7cf2cb85-dd17-4eb5-a5c1-a938840b8ecf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:20:57.087561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:20:57.087561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:20:57.087561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:20:57.087561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:20:57.087561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:20:57.087561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:20:57.087561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:20:57.087561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:20:57.087561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:20:57.087561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:20:57.087561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:20:57.087561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:20:57.087561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:20:57.087561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:20:57.087561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:20:57.088706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:20:57.088706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:20:57.088706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:20:57.088706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:20:57.088706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:20:57.088706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:20:57.088706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:20:57.088706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:20:57.088706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:20:57.088706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:20:57.088706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:20:57.088706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:20:57.088706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:20:57.088706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:20:57.088706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:20:57.088706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:20:57.089772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:20:57.089772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:20:57.089772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:20:57.089772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:20:57.089772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:20:57.089772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:20:57.089772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:20:57.089772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:20:57.089772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:20:57.089772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:20:57.089772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:20:57.089772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:20:57.089772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 579/1149] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:20:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:20:57.094802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46592a5f-98ff-4943-813c-9dffdd587e4b None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:57.190340 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46592a5f-98ff-4943-813c-9dffdd587e4b None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:57.198176 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:57.226974 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8dd15-2fb2-4763-ba2a-3d663cc7a42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:57.227633 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8dd15-2fb2-4763-ba2a-3d663cc7a42d None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:57.227920 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8dd15-2fb2-4763-ba2a-3d663cc7a42d None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:57.228217 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8dd15-2fb2-4763-ba2a-3d663cc7a42d None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:57.228964 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a8dd15-2fb2-4763-ba2a-3d663cc7a42d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:20:57.229278 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a8dd15-2fb2-4763-ba2a-3d663cc7a42d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:57.230644 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a8dd15-2fb2-4763-ba2a-3d663cc7a42d None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:57.238468 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 571/1150] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:20:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:57.244322 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf2cb85-dd17-4eb5-a5c1-a938840b8ecf None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:57.318679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:57.343679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf2cb85-dd17-4eb5-a5c1-a938840b8ecf None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:57.375326 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c80136-ca73-450c-a944-f55dd402d9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:57.376202 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c80136-ca73-450c-a944-f55dd402d9b8 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:57.376496 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c80136-ca73-450c-a944-f55dd402d9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:57.376775 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c80136-ca73-450c-a944-f55dd402d9b8 None admin] PATH_INFO: `/v3/users/0836af286ae743f199e263416caa1daa` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:57.377950 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c80136-ca73-450c-a944-f55dd402d9b8 None admin] RBAC: Authorizing `identity:get_user(user_id=0836af286ae743f199e263416caa1daa)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:57.379955 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c80136-ca73-450c-a944-f55dd402d9b8 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:57.381586 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 580/1151] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:20:57 2022] GET /identity/v3/users/0836af286ae743f199e263416caa1daa => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:57.387141 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8dd15-2fb2-4763-ba2a-3d663cc7a42d None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:57.484301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8dd15-2fb2-4763-ba2a-3d663cc7a42d None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:57.516733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f9990-4f0b-4ce5-8b3f-d59957f0e08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:57.517579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f9990-4f0b-4ce5-8b3f-d59957f0e08e None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:57.517862 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f9990-4f0b-4ce5-8b3f-d59957f0e08e None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:57.518141 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f9990-4f0b-4ce5-8b3f-d59957f0e08e None admin] PATH_INFO: `/v3/projects/a974f54b640140a2b5242a47516196b1` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:57.519296 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f9990-4f0b-4ce5-8b3f-d59957f0e08e None admin] RBAC: Authorizing `identity:get_project(project_id=a974f54b640140a2b5242a47516196b1)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:57.521765 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f9990-4f0b-4ce5-8b3f-d59957f0e08e None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:57.523400 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 572/1152] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:20:57 2022] GET /identity/v3/projects/a974f54b640140a2b5242a47516196b1 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:20:57.529037 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c80136-ca73-450c-a944-f55dd402d9b8 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:20:57.625324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c80136-ca73-450c-a944-f55dd402d9b8 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:20:57.657673 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf3fb06-9825-4758-8aee-b8e03329ef51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:20:57.658543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3fb06-9825-4758-8aee-b8e03329ef51 None admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:57.658826 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3fb06-9825-4758-8aee-b8e03329ef51 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:57.659102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3fb06-9825-4758-8aee-b8e03329ef51 None admin] PATH_INFO: `/v3/projects/a974f54b640140a2b5242a47516196b1/users/0836af286ae743f199e263416caa1daa/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:57.661389 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf3fb06-9825-4758-8aee-b8e03329ef51 None admin] RBAC: Authorizing `identity:create_grant(project_id=a974f54b640140a2b5242a47516196b1, user_id=0836af286ae743f199e263416caa1daa, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:20:57.662792 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf3fb06-9825-4758-8aee-b8e03329ef51 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:20:57.675102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 581/1153] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 12:20:57 2022] PUT /identity/v3/projects/a974f54b640140a2b5242a47516196b1/users/0836af286ae743f199e263416caa1daa/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:20:59.852101 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584d022-94b4-4aa9-9441-3c40dc579804 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:59.852663 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584d022-94b4-4aa9-9441-3c40dc579804 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:59.852754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584d022-94b4-4aa9-9441-3c40dc579804 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:59.854035 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 573/1154] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:20:59.859332 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a53f2-2c14-40c0-a2da-4eefb4bdd6c7 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:20:59.859725 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a53f2-2c14-40c0-a2da-4eefb4bdd6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:20:59.859902 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a53f2-2c14-40c0-a2da-4eefb4bdd6c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:20:59.865575 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:20:59.879683 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-d53a53f2-2c14-40c0-a2da-4eefb4bdd6c7 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:20:59.880906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d53a53f2-2c14-40c0-a2da-4eefb4bdd6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:20:59.946917 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 582/1155] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:20:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:20:59.970094 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584d022-94b4-4aa9-9441-3c40dc579804 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:00.006313 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:21:00.071061 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584d022-94b4-4aa9-9441-3c40dc579804 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:00.103261 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8b938-cc5e-4bdc-bf0f-8453626abd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:00.103974 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8b938-cc5e-4bdc-bf0f-8453626abd6b None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:00.104287 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8b938-cc5e-4bdc-bf0f-8453626abd6b None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:00.104570 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8b938-cc5e-4bdc-bf0f-8453626abd6b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:00.115365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf8b938-cc5e-4bdc-bf0f-8453626abd6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:21:00.122814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-adf8b938-cc5e-4bdc-bf0f-8453626abd6b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:21:00.122814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:21:00.122814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:21:00.122814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:21:00.122814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:21:00.122814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:21:00.122814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:21:00.122814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:21:00.122814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:21:00.122814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:21:00.122814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:21:00.122814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:21:00.122814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:21:00.122814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:21:00.122814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:21:00.122814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:21:00.123951 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:21:00.123951 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:21:00.123951 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:21:00.123951 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:21:00.123951 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:21:00.123951 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:21:00.123951 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:21:00.123951 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:21:00.123951 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:21:00.123951 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:21:00.123951 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:21:00.123951 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:21:00.123951 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:21:00.123951 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:21:00.123951 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:21:00.123951 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:21:00.125039 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:21:00.125039 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:21:00.125039 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:21:00.125039 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:21:00.125039 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:21:00.125039 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:21:00.125039 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:21:00.125039 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:21:00.125039 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:21:00.125039 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:21:00.125039 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:21:00.125039 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:21:00.125039 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 574/1156] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:20:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:21:00.130699 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a53f2-2c14-40c0-a2da-4eefb4bdd6c7 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:00.224905 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a53f2-2c14-40c0-a2da-4eefb4bdd6c7 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:00.252596 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:21:00.258382 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56dc82a-7510-4cb6-ab02-fe20294d51c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:00.259039 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56dc82a-7510-4cb6-ab02-fe20294d51c6 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:00.259324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56dc82a-7510-4cb6-ab02-fe20294d51c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:00.259604 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56dc82a-7510-4cb6-ab02-fe20294d51c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:00.260376 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56dc82a-7510-4cb6-ab02-fe20294d51c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:21:00.260686 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56dc82a-7510-4cb6-ab02-fe20294d51c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:21:00.262045 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56dc82a-7510-4cb6-ab02-fe20294d51c6 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:21:00.270584 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 583/1157] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:21:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:21:00.276419 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8b938-cc5e-4bdc-bf0f-8453626abd6b None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:00.370799 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:21:00.376596 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8b938-cc5e-4bdc-bf0f-8453626abd6b None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:00.409887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbffb109-9339-4e70-892c-4ebf5233e2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:00.411006 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffb109-9339-4e70-892c-4ebf5233e2ce None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:00.411284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffb109-9339-4e70-892c-4ebf5233e2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:00.411559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffb109-9339-4e70-892c-4ebf5233e2ce None admin] PATH_INFO: `/v3/users/0836af286ae743f199e263416caa1daa` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:00.412745 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbffb109-9339-4e70-892c-4ebf5233e2ce None admin] RBAC: Authorizing `identity:get_user(user_id=0836af286ae743f199e263416caa1daa)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:21:00.414174 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbffb109-9339-4e70-892c-4ebf5233e2ce None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:21:00.415758 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 575/1158] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:21:00 2022] GET /identity/v3/users/0836af286ae743f199e263416caa1daa => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:21:00.421476 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56dc82a-7510-4cb6-ab02-fe20294d51c6 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:00.517975 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56dc82a-7510-4cb6-ab02-fe20294d51c6 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:00.550086 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae878c64-277f-49ad-975c-9ae7745f6a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:00.551379 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae878c64-277f-49ad-975c-9ae7745f6a54 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:00.551743 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae878c64-277f-49ad-975c-9ae7745f6a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:00.552025 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae878c64-277f-49ad-975c-9ae7745f6a54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:00.610270 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:21:00.618301 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae878c64-277f-49ad-975c-9ae7745f6a54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:21:00.619742 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae878c64-277f-49ad-975c-9ae7745f6a54 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:21:00.714537 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 584/1159] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:21:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 12:21:00.720334 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbffb109-9339-4e70-892c-4ebf5233e2ce None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:00.815898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbffb109-9339-4e70-892c-4ebf5233e2ce None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:00.848892 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440b417b-6d7b-4fc7-ba09-c787cdb8d180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:00.849723 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b417b-6d7b-4fc7-ba09-c787cdb8d180 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:00.850006 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b417b-6d7b-4fc7-ba09-c787cdb8d180 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:00.850282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440b417b-6d7b-4fc7-ba09-c787cdb8d180 None admin] PATH_INFO: `/v3/projects/a974f54b640140a2b5242a47516196b1` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:00.851448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440b417b-6d7b-4fc7-ba09-c787cdb8d180 None admin] RBAC: Authorizing `identity:get_project(project_id=a974f54b640140a2b5242a47516196b1)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:21:00.852903 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440b417b-6d7b-4fc7-ba09-c787cdb8d180 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:21:00.854540 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 576/1160] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:21:00 2022] GET /identity/v3/projects/a974f54b640140a2b5242a47516196b1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:21:00.860427 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae878c64-277f-49ad-975c-9ae7745f6a54 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:00.949949 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:21:00.956496 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae878c64-277f-49ad-975c-9ae7745f6a54 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:00.989756 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba912e-7a97-4ecc-8d20-f931995c136c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:00.990321 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba912e-7a97-4ecc-8d20-f931995c136c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:00.990600 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba912e-7a97-4ecc-8d20-f931995c136c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:00.990874 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba912e-7a97-4ecc-8d20-f931995c136c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:00.991681 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba912e-7a97-4ecc-8d20-f931995c136c None admin] RBAC: Adding query filter params (role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8, scope.project.id=a974f54b640140a2b5242a47516196b1, user.id=0836af286ae743f199e263416caa1daa) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:21:00.991988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba912e-7a97-4ecc-8d20-f931995c136c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:21:00.993426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba912e-7a97-4ecc-8d20-f931995c136c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:21:00.998981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 585/1161] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:21:00 2022] GET /identity/v3/role_assignments?user.id=0836af286ae743f199e263416caa1daa&scope.project.id=a974f54b640140a2b5242a47516196b1&role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:21:03.213764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae871c-a827-4f58-a65d-6f7c29f816d5 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:03.214388 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae871c-a827-4f58-a65d-6f7c29f816d5 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:03.214388 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae871c-a827-4f58-a65d-6f7c29f816d5 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:03.215648 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 577/1162] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:21:03.220776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeea65e-f8f5-473b-b006-d4ae3da88365 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:03.221216 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeea65e-f8f5-473b-b006-d4ae3da88365 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:03.221326 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeea65e-f8f5-473b-b006-d4ae3da88365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:03.237677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-ffeea65e-f8f5-473b-b006-d4ae3da88365 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:21:03.239004 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ffeea65e-f8f5-473b-b006-d4ae3da88365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:21:03.302435 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 586/1163] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:21:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:21:03.325181 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae871c-a827-4f58-a65d-6f7c29f816d5 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:03.356161 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:21:03.425834 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae871c-a827-4f58-a65d-6f7c29f816d5 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:03.466378 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62113fb-8781-4afc-a581-56cc4df03941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:03.467026 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62113fb-8781-4afc-a581-56cc4df03941 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:03.467314 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62113fb-8781-4afc-a581-56cc4df03941 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:03.467596 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62113fb-8781-4afc-a581-56cc4df03941 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:03.532143 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62113fb-8781-4afc-a581-56cc4df03941 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:21:03.533623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62113fb-8781-4afc-a581-56cc4df03941 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:21:03.562097 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:21:03.631054 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 578/1164] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:21:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 12:21:03.637165 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeea65e-f8f5-473b-b006-d4ae3da88365 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:03.721661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:21:03.733476 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeea65e-f8f5-473b-b006-d4ae3da88365 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:03.766707 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3586def6-8e42-44c3-9abc-6a1b552b1b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:03.767502 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586def6-8e42-44c3-9abc-6a1b552b1b1e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:03.767783 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586def6-8e42-44c3-9abc-6a1b552b1b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:03.768057 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586def6-8e42-44c3-9abc-6a1b552b1b1e None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:03.774888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3586def6-8e42-44c3-9abc-6a1b552b1b1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:21:03.776466 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3586def6-8e42-44c3-9abc-6a1b552b1b1e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:21:03.790421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-3586def6-8e42-44c3-9abc-6a1b552b1b1e None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 23 12:21:03.790421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:21:03.790421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 23 12:21:03.790421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 23 12:21:03.790421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:21:03.790421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:21:03.790421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:21:03.790421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:21:03.790421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 23 12:21:03.790421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:21:03.790421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:21:03.790421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:21:03.790421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:21:03.790421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:21:03.790421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:21:03.790421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:21:03.790421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:21:03.791688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:21:03.791688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:21:03.791688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:21:03.791688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:21:03.791688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:21:03.791688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:21:03.791688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:21:03.791688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:21:03.791688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:21:03.791688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:21:03.791688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:21:03.791688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:21:03.791688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:21:03.791688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:21:03.791688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:21:03.791688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:21:03.792675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:21:03.792675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:21:03.792675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:21:03.792675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:21:03.792675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:21:03.792675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:21:03.792675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:21:03.792675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:21:03.792675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:21:03.792675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:21:03.792675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 23 12:21:03.792675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:21:03.792675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 23 12:21:03.792675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:21:03.792675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 587/1165] 10.222.0.16 () {56 vars in 1117 bytes} [Mon May 23 12:21:03 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:21:03.797635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62113fb-8781-4afc-a581-56cc4df03941 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:03.894300 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62113fb-8781-4afc-a581-56cc4df03941 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:03.923158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:21:03.928970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b925722-523f-4a90-a498-cb757074010d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:03.929630 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b925722-523f-4a90-a498-cb757074010d None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:03.929913 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b925722-523f-4a90-a498-cb757074010d None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:03.930190 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b925722-523f-4a90-a498-cb757074010d None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:03.930906 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b925722-523f-4a90-a498-cb757074010d None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:21:03.931213 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b925722-523f-4a90-a498-cb757074010d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:21:03.932617 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b925722-523f-4a90-a498-cb757074010d None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:21:03.939587 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 579/1166] 10.222.0.16 () {56 vars in 1115 bytes} [Mon May 23 12:21:03 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:21:06.133688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a21c3-edf0-412f-b6a2-9b55891277ee None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:06.134223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a21c3-edf0-412f-b6a2-9b55891277ee None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:06.134223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a21c3-edf0-412f-b6a2-9b55891277ee None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:06.135582 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 588/1167] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:21:06.140849 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b70dd1-dfde-45a1-b73d-b25d75a41dc8 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:06.141241 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b70dd1-dfde-45a1-b73d-b25d75a41dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:06.141407 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b70dd1-dfde-45a1-b73d-b25d75a41dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:06.158067 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-f1b70dd1-dfde-45a1-b73d-b25d75a41dc8 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:21:06.159236 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-f1b70dd1-dfde-45a1-b73d-b25d75a41dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:21:06.224827 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 580/1168] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:21:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:21:06.247584 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a21c3-edf0-412f-b6a2-9b55891277ee None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:06.346588 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a21c3-edf0-412f-b6a2-9b55891277ee None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:06.380563 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108668f-6045-46fd-8e8c-740c0428a613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:06.381206 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108668f-6045-46fd-8e8c-740c0428a613 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:06.381486 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108668f-6045-46fd-8e8c-740c0428a613 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:06.381762 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108668f-6045-46fd-8e8c-740c0428a613 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:06.382504 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c108668f-6045-46fd-8e8c-740c0428a613 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:21:06.383854 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c108668f-6045-46fd-8e8c-740c0428a613 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:21:06.398888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 589/1169] 10.222.0.16 () {60 vars in 1138 bytes} [Mon May 23 12:21:06 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:21:08.602869 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac19ed1-b996-41f1-b3f0-660da4fb88b9 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:08.603435 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac19ed1-b996-41f1-b3f0-660da4fb88b9 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:08.603435 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac19ed1-b996-41f1-b3f0-660da4fb88b9 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:08.604803 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 581/1170] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:21:08.609914 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042fc06-aae0-4f13-9523-f71b87b9be43 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:08.610294 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042fc06-aae0-4f13-9523-f71b87b9be43 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:08.610487 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042fc06-aae0-4f13-9523-f71b87b9be43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:08.628100 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-2042fc06-aae0-4f13-9523-f71b87b9be43 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:21:08.629682 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-2042fc06-aae0-4f13-9523-f71b87b9be43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:21:08.631618 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:21:08.697045 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 590/1171] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:21:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:21:08.719645 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac19ed1-b996-41f1-b3f0-660da4fb88b9 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:08.821605 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac19ed1-b996-41f1-b3f0-660da4fb88b9 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:08.846299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:21:08.856105 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c8a995-6b88-4efa-b104-bf3b23e3666c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:08.856925 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8a995-6b88-4efa-b104-bf3b23e3666c None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:08.857212 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8a995-6b88-4efa-b104-bf3b23e3666c None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:08.857493 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8a995-6b88-4efa-b104-bf3b23e3666c None admin] PATH_INFO: `/v3/domains/dd248e2e9a7248aebebe97f7c8132cf2` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:08.867926 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c8a995-6b88-4efa-b104-bf3b23e3666c None admin] RBAC: Authorizing `identity:get_domain(domain_id=dd248e2e9a7248aebebe97f7c8132cf2)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:21:08.869409 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c8a995-6b88-4efa-b104-bf3b23e3666c None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:21:08.871033 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 582/1172] 10.222.0.16 () {56 vars in 1183 bytes} [Mon May 23 12:21:08 2022] GET /identity/v3/domains/dd248e2e9a7248aebebe97f7c8132cf2 => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:21:08.876829 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042fc06-aae0-4f13-9523-f71b87b9be43 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:08.972910 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042fc06-aae0-4f13-9523-f71b87b9be43 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:09.005425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b7e8f-8314-4c82-8874-124fb9b986b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:09.006071 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b7e8f-8314-4c82-8874-124fb9b986b5 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:09.006355 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b7e8f-8314-4c82-8874-124fb9b986b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:09.006632 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b7e8f-8314-4c82-8874-124fb9b986b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:09.007378 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b7e8f-8314-4c82-8874-124fb9b986b5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:21:09.008830 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b7e8f-8314-4c82-8874-124fb9b986b5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:21:09.016495 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:21:09.036063 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 591/1173] 10.222.0.16 () {60 vars in 1141 bytes} [Mon May 23 12:21:08 2022] POST /identity/v3/projects => generated 361 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:21:11.263519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506f388-7a28-4f7b-84a5-101ac39d7a7e None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:11.264067 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506f388-7a28-4f7b-84a5-101ac39d7a7e None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:11.264067 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506f388-7a28-4f7b-84a5-101ac39d7a7e None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:11.265464 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 583/1174] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:21:11.270612 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc35e6-e552-4143-8aa1-d71bbcab3e41 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:11.271046 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc35e6-e552-4143-8aa1-d71bbcab3e41 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:11.271177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc35e6-e552-4143-8aa1-d71bbcab3e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:11.287595 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-45dc35e6-e552-4143-8aa1-d71bbcab3e41 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:21:11.288745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-45dc35e6-e552-4143-8aa1-d71bbcab3e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:21:11.355080 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 592/1175] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:21:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:21:11.381118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506f388-7a28-4f7b-84a5-101ac39d7a7e None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:11.480407 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506f388-7a28-4f7b-84a5-101ac39d7a7e None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:11.513761 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af688043-07af-428e-9611-0909188e5a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:11.514603 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af688043-07af-428e-9611-0909188e5a7b None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:11.514887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af688043-07af-428e-9611-0909188e5a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:11.515165 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af688043-07af-428e-9611-0909188e5a7b None admin] PATH_INFO: `/v3/domains/dd248e2e9a7248aebebe97f7c8132cf2` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:11.516375 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af688043-07af-428e-9611-0909188e5a7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=dd248e2e9a7248aebebe97f7c8132cf2)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:21:11.517807 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af688043-07af-428e-9611-0909188e5a7b None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:21:11.519442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 584/1176] 10.222.0.16 () {56 vars in 1183 bytes} [Mon May 23 12:21:11 2022] GET /identity/v3/domains/dd248e2e9a7248aebebe97f7c8132cf2 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:21:11.525048 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc35e6-e552-4143-8aa1-d71bbcab3e41 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:11.620444 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc35e6-e552-4143-8aa1-d71bbcab3e41 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:11.639426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:21:11.653509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d62c00-061b-4014-8f6e-fafebd5f1816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:11.654165 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d62c00-061b-4014-8f6e-fafebd5f1816 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:11.654445 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d62c00-061b-4014-8f6e-fafebd5f1816 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:11.654748 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d62c00-061b-4014-8f6e-fafebd5f1816 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:11.655451 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d62c00-061b-4014-8f6e-fafebd5f1816 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:21:11.656882 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d62c00-061b-4014-8f6e-fafebd5f1816 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:21:11.677067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 593/1177] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 12:21:11 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:21:13.917968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e8df5-2c17-441a-9954-2a24c0c931a1 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:13.918517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e8df5-2c17-441a-9954-2a24c0c931a1 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:13.918517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e8df5-2c17-441a-9954-2a24c0c931a1 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:13.919902 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 585/1178] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:21:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:21:13.926164 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5784f112-d29f-435a-a9f4-aafb5746751f None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:13.926590 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5784f112-d29f-435a-a9f4-aafb5746751f None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:13.926789 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5784f112-d29f-435a-a9f4-aafb5746751f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:13.943292 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-5784f112-d29f-435a-a9f4-aafb5746751f None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:21:13.944484 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5784f112-d29f-435a-a9f4-aafb5746751f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:21:14.008765 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 594/1179] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:21:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:21:14.031457 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e8df5-2c17-441a-9954-2a24c0c931a1 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:14.088034 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:21:14.131787 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e8df5-2c17-441a-9954-2a24c0c931a1 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:14.164861 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a8ac3-a786-4faa-868d-cbe4c6261402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:14.165637 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a8ac3-a786-4faa-868d-cbe4c6261402 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:14.165919 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a8ac3-a786-4faa-868d-cbe4c6261402 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:14.166193 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a8ac3-a786-4faa-868d-cbe4c6261402 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:14.176023 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754a8ac3-a786-4faa-868d-cbe4c6261402 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:21:14.183496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-754a8ac3-a786-4faa-868d-cbe4c6261402 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:21:14.183496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:21:14.183496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:21:14.183496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:21:14.183496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:21:14.183496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:21:14.183496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:21:14.183496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:21:14.183496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:21:14.183496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:21:14.183496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:21:14.183496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:21:14.183496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:21:14.183496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:21:14.183496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:21:14.183496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:21:14.184656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:21:14.184656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:21:14.184656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:21:14.184656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:21:14.184656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:21:14.184656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:21:14.184656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:21:14.184656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:21:14.184656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:21:14.184656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:21:14.184656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:21:14.184656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:21:14.184656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:21:14.184656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:21:14.184656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:21:14.184656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:21:14.185731 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:21:14.185731 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:21:14.185731 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:21:14.185731 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:21:14.185731 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:21:14.185731 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:21:14.185731 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:21:14.185731 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:21:14.185731 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:21:14.185731 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:21:14.185731 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:21:14.185731 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:21:14.185731 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 586/1180] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:21:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:21:14.190898 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5784f112-d29f-435a-a9f4-aafb5746751f None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:14.287525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5784f112-d29f-435a-a9f4-aafb5746751f None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:14.294532 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:21:14.322887 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed07c6c7-bc53-41ce-a203-2d43af7a6157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:14.323548 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07c6c7-bc53-41ce-a203-2d43af7a6157 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:14.323833 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07c6c7-bc53-41ce-a203-2d43af7a6157 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:14.324110 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07c6c7-bc53-41ce-a203-2d43af7a6157 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:14.324875 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed07c6c7-bc53-41ce-a203-2d43af7a6157 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:21:14.325180 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed07c6c7-bc53-41ce-a203-2d43af7a6157 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:21:14.326833 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed07c6c7-bc53-41ce-a203-2d43af7a6157 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:21:14.335261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 595/1181] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:21:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:21:14.340956 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a8ac3-a786-4faa-868d-cbe4c6261402 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:14.454420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a8ac3-a786-4faa-868d-cbe4c6261402 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:14.468181 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:21:14.487554 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37c6b5-1de4-414e-8241-c62de553ce9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:14.488749 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37c6b5-1de4-414e-8241-c62de553ce9a None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:14.489033 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37c6b5-1de4-414e-8241-c62de553ce9a None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:14.489316 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d37c6b5-1de4-414e-8241-c62de553ce9a None admin] PATH_INFO: `/v3/users/242e3ed165de469f8493dc24db698b67` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:14.502556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37c6b5-1de4-414e-8241-c62de553ce9a None admin] RBAC: Authorizing `identity:get_user(user_id=242e3ed165de469f8493dc24db698b67)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:21:14.505132 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d37c6b5-1de4-414e-8241-c62de553ce9a None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:21:14.506924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 587/1182] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:21:14 2022] GET /identity/v3/users/242e3ed165de469f8493dc24db698b67 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:21:14.512689 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed07c6c7-bc53-41ce-a203-2d43af7a6157 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:14.610947 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed07c6c7-bc53-41ce-a203-2d43af7a6157 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:14.643472 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435432db-ba90-43ff-9bbd-7ca865efa5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:14.644785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435432db-ba90-43ff-9bbd-7ca865efa5bd None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:14.645073 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435432db-ba90-43ff-9bbd-7ca865efa5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:14.645354 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435432db-ba90-43ff-9bbd-7ca865efa5bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:14.677900 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:21:14.710834 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435432db-ba90-43ff-9bbd-7ca865efa5bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:21:14.712246 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435432db-ba90-43ff-9bbd-7ca865efa5bd None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:21:14.829814 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 596/1183] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:21:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 12:21:14.835659 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37c6b5-1de4-414e-8241-c62de553ce9a None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:14.932411 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d37c6b5-1de4-414e-8241-c62de553ce9a None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:14.965642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72329fd4-07f2-4ae3-8da2-a1f77858d0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:14.966420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72329fd4-07f2-4ae3-8da2-a1f77858d0d7 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:14.966700 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72329fd4-07f2-4ae3-8da2-a1f77858d0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:14.966973 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72329fd4-07f2-4ae3-8da2-a1f77858d0d7 None admin] PATH_INFO: `/v3/projects/649d99dd44a14f7caff504418f77d63e` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:14.968122 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72329fd4-07f2-4ae3-8da2-a1f77858d0d7 None admin] RBAC: Authorizing `identity:get_project(project_id=649d99dd44a14f7caff504418f77d63e)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:21:14.969579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72329fd4-07f2-4ae3-8da2-a1f77858d0d7 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:21:14.971213 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 588/1184] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:21:14 2022] GET /identity/v3/projects/649d99dd44a14f7caff504418f77d63e => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:21:14.977324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435432db-ba90-43ff-9bbd-7ca865efa5bd None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:15.041654 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:21:15.074921 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435432db-ba90-43ff-9bbd-7ca865efa5bd None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:15.107224 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2dc187-a738-4628-9226-9c2e07d350cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:15.107866 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2dc187-a738-4628-9226-9c2e07d350cb None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:15.108148 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2dc187-a738-4628-9226-9c2e07d350cb None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:15.108458 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2dc187-a738-4628-9226-9c2e07d350cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:15.109275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2dc187-a738-4628-9226-9c2e07d350cb None admin] RBAC: Adding query filter params (role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8, scope.project.id=649d99dd44a14f7caff504418f77d63e, user.id=242e3ed165de469f8493dc24db698b67) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:21:15.109586 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2dc187-a738-4628-9226-9c2e07d350cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:21:15.111001 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2dc187-a738-4628-9226-9c2e07d350cb None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:21:15.117012 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 597/1185] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:21:14 2022] GET /identity/v3/role_assignments?user.id=242e3ed165de469f8493dc24db698b67&scope.project.id=649d99dd44a14f7caff504418f77d63e&role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:21:17.414625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f672f12-ee8a-4bcb-b78a-560dafb68ed6 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:17.415145 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f672f12-ee8a-4bcb-b78a-560dafb68ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:17.415256 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f672f12-ee8a-4bcb-b78a-560dafb68ed6 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:17.416547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 589/1186] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:21:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:21:17.421714 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acf2d1-e851-4b3f-9297-dcbb825711f0 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:17.422136 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acf2d1-e851-4b3f-9297-dcbb825711f0 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:17.422279 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acf2d1-e851-4b3f-9297-dcbb825711f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:17.439055 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a0acf2d1-e851-4b3f-9297-dcbb825711f0 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:21:17.440256 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a0acf2d1-e851-4b3f-9297-dcbb825711f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:21:17.505561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 598/1187] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:21:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:21:17.527384 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f672f12-ee8a-4bcb-b78a-560dafb68ed6 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:17.570307 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:21:17.627994 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f672f12-ee8a-4bcb-b78a-560dafb68ed6 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:17.661402 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6572cd5f-6e96-4150-b924-141fc1ab5037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:17.662168 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572cd5f-6e96-4150-b924-141fc1ab5037 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:17.662449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572cd5f-6e96-4150-b924-141fc1ab5037 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:17.662722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6572cd5f-6e96-4150-b924-141fc1ab5037 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:17.672466 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6572cd5f-6e96-4150-b924-141fc1ab5037 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:21:17.680142 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-6572cd5f-6e96-4150-b924-141fc1ab5037 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:21:17.680142 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:21:17.680142 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:21:17.680142 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:21:17.680142 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:21:17.680142 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:21:17.680142 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:21:17.680142 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:21:17.680142 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:21:17.680142 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:21:17.680142 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:21:17.680142 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:21:17.680142 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:21:17.680142 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:21:17.680142 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:21:17.680142 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:21:17.681091 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:21:17.681091 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:21:17.681091 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:21:17.681091 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:21:17.681091 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:21:17.681091 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:21:17.681091 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:21:17.681091 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:21:17.681091 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:21:17.681091 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:21:17.681091 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:21:17.681091 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:21:17.681091 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:21:17.681091 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:21:17.681091 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:21:17.681091 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:21:17.682036 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:21:17.682036 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:21:17.682036 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:21:17.682036 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:21:17.682036 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:21:17.682036 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:21:17.682036 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:21:17.682036 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:21:17.682036 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:21:17.682036 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:21:17.682036 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:21:17.682036 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:21:17.682036 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 590/1188] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:21:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:21:17.687241 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0acf2d1-e851-4b3f-9297-dcbb825711f0 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:17.746336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:21:17.783596 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0acf2d1-e851-4b3f-9297-dcbb825711f0 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:17.816381 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba28fb-91e0-404f-b56c-6678f557937c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:17.817038 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba28fb-91e0-404f-b56c-6678f557937c None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:17.817321 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba28fb-91e0-404f-b56c-6678f557937c None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:17.817600 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edba28fb-91e0-404f-b56c-6678f557937c None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:17.818593 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba28fb-91e0-404f-b56c-6678f557937c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:21:17.818901 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba28fb-91e0-404f-b56c-6678f557937c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:21:17.820274 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edba28fb-91e0-404f-b56c-6678f557937c None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:21:17.828005 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 599/1189] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:21:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:21:17.834235 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6572cd5f-6e96-4150-b924-141fc1ab5037 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:17.928456 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:21:17.931391 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6572cd5f-6e96-4150-b924-141fc1ab5037 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:17.964767 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b5409-b078-4e0f-a0d3-4de105d68cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:17.965628 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b5409-b078-4e0f-a0d3-4de105d68cfb None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:17.965915 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b5409-b078-4e0f-a0d3-4de105d68cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:17.966196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b5409-b078-4e0f-a0d3-4de105d68cfb None admin] PATH_INFO: `/v3/users/242e3ed165de469f8493dc24db698b67` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:17.967373 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b5409-b078-4e0f-a0d3-4de105d68cfb None admin] RBAC: Authorizing `identity:get_user(user_id=242e3ed165de469f8493dc24db698b67)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:21:17.968848 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b5409-b078-4e0f-a0d3-4de105d68cfb None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:21:17.970440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 591/1190] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:21:17 2022] GET /identity/v3/users/242e3ed165de469f8493dc24db698b67 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:21:17.976001 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba28fb-91e0-404f-b56c-6678f557937c None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:18.075107 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edba28fb-91e0-404f-b56c-6678f557937c None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:18.098637 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:21:18.109482 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d689c1-0faa-4154-9607-789ded459c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:18.110366 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d689c1-0faa-4154-9607-789ded459c11 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:18.110654 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d689c1-0faa-4154-9607-789ded459c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:18.110943 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d689c1-0faa-4154-9607-789ded459c11 None admin] PATH_INFO: `/v3/projects/649d99dd44a14f7caff504418f77d63e` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:18.112099 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d689c1-0faa-4154-9607-789ded459c11 None admin] RBAC: Authorizing `identity:get_project(project_id=649d99dd44a14f7caff504418f77d63e)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:21:18.114132 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d689c1-0faa-4154-9607-789ded459c11 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:21:18.115808 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 600/1191] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:21:17 2022] GET /identity/v3/projects/649d99dd44a14f7caff504418f77d63e => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:21:18.121901 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b5409-b078-4e0f-a0d3-4de105d68cfb None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:18.220565 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b5409-b078-4e0f-a0d3-4de105d68cfb None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:18.253496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3b61f-51d0-45d7-ae7d-dfc54251609d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:18.254379 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3b61f-51d0-45d7-ae7d-dfc54251609d None admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:18.255000 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3b61f-51d0-45d7-ae7d-dfc54251609d None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:18.255289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3b61f-51d0-45d7-ae7d-dfc54251609d None admin] PATH_INFO: `/v3/projects/649d99dd44a14f7caff504418f77d63e/users/242e3ed165de469f8493dc24db698b67/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:18.257287 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3b61f-51d0-45d7-ae7d-dfc54251609d None admin] RBAC: Authorizing `identity:create_grant(project_id=649d99dd44a14f7caff504418f77d63e, user_id=242e3ed165de469f8493dc24db698b67, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:21:18.258721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3b61f-51d0-45d7-ae7d-dfc54251609d None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:21:18.272136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 592/1192] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 12:21:18 2022] PUT /identity/v3/projects/649d99dd44a14f7caff504418f77d63e/users/242e3ed165de469f8493dc24db698b67/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:21:20.478864 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eba43f-359b-405e-bca0-34b617949b11 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:20.479427 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eba43f-359b-405e-bca0-34b617949b11 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:20.479427 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eba43f-359b-405e-bca0-34b617949b11 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:20.480803 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 601/1193] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:21:20.486043 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9569ecf-dfa6-4aaa-8d77-50fdd69d8f22 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:20.486430 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9569ecf-dfa6-4aaa-8d77-50fdd69d8f22 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:20.486639 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9569ecf-dfa6-4aaa-8d77-50fdd69d8f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:20.504433 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-b9569ecf-dfa6-4aaa-8d77-50fdd69d8f22 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:21:20.505618 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-b9569ecf-dfa6-4aaa-8d77-50fdd69d8f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:21:20.527694 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:21:20.572332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 593/1194] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:21:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:21:20.595083 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eba43f-359b-405e-bca0-34b617949b11 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:20.695031 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eba43f-359b-405e-bca0-34b617949b11 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:20.735741 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2e4db-f69e-4f91-8858-fdb378c043a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:20.736487 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2e4db-f69e-4f91-8858-fdb378c043a1 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:20.736773 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2e4db-f69e-4f91-8858-fdb378c043a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:20.737047 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2e4db-f69e-4f91-8858-fdb378c043a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:20.750146 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb2e4db-f69e-4f91-8858-fdb378c043a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:21:20.757836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-fbb2e4db-f69e-4f91-8858-fdb378c043a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:21:20.757836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:21:20.757836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:21:20.757836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:21:20.757836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:21:20.757836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:21:20.757836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:21:20.757836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:21:20.757836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:21:20.757836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:21:20.757836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:21:20.757836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:21:20.757836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:21:20.757836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:21:20.757836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:21:20.757836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:21:20.758960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:21:20.758960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:21:20.758960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:21:20.758960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:21:20.758960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:21:20.758960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:21:20.758960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:21:20.758960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:21:20.758960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:21:20.758960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:21:20.758960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:21:20.758960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:21:20.758960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:21:20.758960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:21:20.758960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:21:20.758960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:21:20.759852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:21:20.759852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:21:20.759852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:21:20.759852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:21:20.759852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:21:20.759852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:21:20.759852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:21:20.759852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:21:20.759852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:21:20.759852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:21:20.759852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:21:20.759852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:21:20.759852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 602/1195] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:21:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:21:20.765051 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9569ecf-dfa6-4aaa-8d77-50fdd69d8f22 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:20.862829 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9569ecf-dfa6-4aaa-8d77-50fdd69d8f22 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:20.895988 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df325e2-5dd5-4227-aa40-31aa79b1dfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:20.896671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df325e2-5dd5-4227-aa40-31aa79b1dfd2 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:20.896956 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df325e2-5dd5-4227-aa40-31aa79b1dfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:20.897230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df325e2-5dd5-4227-aa40-31aa79b1dfd2 None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:20.897968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df325e2-5dd5-4227-aa40-31aa79b1dfd2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:21:20.898275 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df325e2-5dd5-4227-aa40-31aa79b1dfd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:21:20.899639 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df325e2-5dd5-4227-aa40-31aa79b1dfd2 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:21:20.907400 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 594/1196] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:21:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:21:20.914021 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2e4db-f69e-4f91-8858-fdb378c043a1 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:20.965731 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:21:21.011864 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2e4db-f69e-4f91-8858-fdb378c043a1 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:21.045311 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05adcb62-294d-415e-89fe-fe8dba36a6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:21.046180 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05adcb62-294d-415e-89fe-fe8dba36a6c6 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:21.046465 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05adcb62-294d-415e-89fe-fe8dba36a6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:21.046742 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05adcb62-294d-415e-89fe-fe8dba36a6c6 None admin] PATH_INFO: `/v3/users/242e3ed165de469f8493dc24db698b67` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:21.047933 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05adcb62-294d-415e-89fe-fe8dba36a6c6 None admin] RBAC: Authorizing `identity:get_user(user_id=242e3ed165de469f8493dc24db698b67)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:21:21.049431 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05adcb62-294d-415e-89fe-fe8dba36a6c6 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:21:21.051053 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 603/1197] 10.222.0.16 () {56 vars in 1177 bytes} [Mon May 23 12:21:20 2022] GET /identity/v3/users/242e3ed165de469f8493dc24db698b67 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:21:21.056806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df325e2-5dd5-4227-aa40-31aa79b1dfd2 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:21.087182 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:21:21.167380 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df325e2-5dd5-4227-aa40-31aa79b1dfd2 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:21.202044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e10e1-cf0c-4071-8420-e653b2b55d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:21.202690 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e10e1-cf0c-4071-8420-e653b2b55d6a None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:21.202981 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e10e1-cf0c-4071-8420-e653b2b55d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:21.203264 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0e10e1-cf0c-4071-8420-e653b2b55d6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:21.268726 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0e10e1-cf0c-4071-8420-e653b2b55d6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:21:21.270661 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0e10e1-cf0c-4071-8420-e653b2b55d6a None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:21:21.290268 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:21:21.366530 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 595/1198] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:21:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 12:21:21.372448 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05adcb62-294d-415e-89fe-fe8dba36a6c6 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:21.472585 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05adcb62-294d-415e-89fe-fe8dba36a6c6 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:21.491347 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:21:21.505665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb16ea3-2a5a-42a6-9eab-a457d466c5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:21.506525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb16ea3-2a5a-42a6-9eab-a457d466c5b2 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:21.506810 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb16ea3-2a5a-42a6-9eab-a457d466c5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:21.507093 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb16ea3-2a5a-42a6-9eab-a457d466c5b2 None admin] PATH_INFO: `/v3/projects/649d99dd44a14f7caff504418f77d63e` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:21.508283 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb16ea3-2a5a-42a6-9eab-a457d466c5b2 None admin] RBAC: Authorizing `identity:get_project(project_id=649d99dd44a14f7caff504418f77d63e)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:21:21.509719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb16ea3-2a5a-42a6-9eab-a457d466c5b2 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:21:21.511443 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 604/1199] 10.222.0.16 () {56 vars in 1186 bytes} [Mon May 23 12:21:21 2022] GET /identity/v3/projects/649d99dd44a14f7caff504418f77d63e => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:21:21.517512 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e10e1-cf0c-4071-8420-e653b2b55d6a None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:21:21.614125 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0e10e1-cf0c-4071-8420-e653b2b55d6a None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:21:21.633845 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:21:21.647667 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455d2346-7f60-40bd-8602-45f80f5148d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:21:21.648342 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d2346-7f60-40bd-8602-45f80f5148d5 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:21:21.648631 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d2346-7f60-40bd-8602-45f80f5148d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:21:21.648913 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d2346-7f60-40bd-8602-45f80f5148d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:21:21.649740 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455d2346-7f60-40bd-8602-45f80f5148d5 None admin] RBAC: Adding query filter params (role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8, scope.project.id=649d99dd44a14f7caff504418f77d63e, user.id=242e3ed165de469f8493dc24db698b67) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:21:21.650056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455d2346-7f60-40bd-8602-45f80f5148d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:21:21.651489 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455d2346-7f60-40bd-8602-45f80f5148d5 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:21:21.657467 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 596/1200] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:21:21 2022] GET /identity/v3/role_assignments?user.id=242e3ed165de469f8493dc24db698b67&scope.project.id=649d99dd44a14f7caff504418f77d63e&role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:21:31.501066 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=6 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:21:31.654907 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=8 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:22.141030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fb3f9-fb4a-4e81-8bb4-75b994703836 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:22.141030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fb3f9-fb4a-4e81-8bb4-75b994703836 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:22.141030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fb3f9-fb4a-4e81-8bb4-75b994703836 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:22.141030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 605/1201] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:22:22.146967 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769e156-a96f-4c4f-af4a-81545fb19f6d None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:22.147775 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769e156-a96f-4c4f-af4a-81545fb19f6d None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:22.148055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769e156-a96f-4c4f-af4a-81545fb19f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:22.167027 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-4769e156-a96f-4c4f-af4a-81545fb19f6d None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:22:22.168263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-4769e156-a96f-4c4f-af4a-81545fb19f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:22:22.265456 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 597/1202] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:22:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:22:22.294494 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3fb3f9-fb4a-4e81-8bb4-75b994703836 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:22.427464 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3fb3f9-fb4a-4e81-8bb4-75b994703836 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:22.468475 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af448bd5-f62a-4d3f-be7d-04392f462dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:22.469315 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af448bd5-f62a-4d3f-be7d-04392f462dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:22.469648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af448bd5-f62a-4d3f-be7d-04392f462dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:22.470103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af448bd5-f62a-4d3f-be7d-04392f462dc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:22.482501 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af448bd5-f62a-4d3f-be7d-04392f462dc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:22.484229 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af448bd5-f62a-4d3f-be7d-04392f462dc8 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:22.508148 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-af448bd5-f62a-4d3f-be7d-04392f462dc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:22:22.508148 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:22.508148 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:22.508148 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:22.508148 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:22.508148 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:22.508148 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:22.508148 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:22.508148 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:22.508148 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:22.508148 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:22.508148 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:22.508148 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:22:22.508148 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:22:22.508148 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:22:22.509247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:22:22.509247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:22.509247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:22.509247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:22.509247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:22.509247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:22.509247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:22.509247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:22.509247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:22:22.509247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:22.509247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:22.509247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:22.509247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:22.509247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:22.509247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:22.509247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:22.509247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:22.516509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:22:22.516509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:22:22.516509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:22:22.516509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:22:22.516509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 606/1203] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:22:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:22:22.517450 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769e156-a96f-4c4f-af4a-81545fb19f6d None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:22.644333 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769e156-a96f-4c4f-af4a-81545fb19f6d None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:22.678710 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:22.687180 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed2a6e3-5965-45ce-a078-d9ff3e0ea4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:22.687924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2a6e3-5965-45ce-a078-d9ff3e0ea4cb None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:22.688292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2a6e3-5965-45ce-a078-d9ff3e0ea4cb None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:22.688536 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2a6e3-5965-45ce-a078-d9ff3e0ea4cb None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:22.689993 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed2a6e3-5965-45ce-a078-d9ff3e0ea4cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:22.690413 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed2a6e3-5965-45ce-a078-d9ff3e0ea4cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:22.691914 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed2a6e3-5965-45ce-a078-d9ff3e0ea4cb None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:22.705368 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 598/1204] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:22:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:22.711450 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af448bd5-f62a-4d3f-be7d-04392f462dc8 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:22.797202 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:22.882655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af448bd5-f62a-4d3f-be7d-04392f462dc8 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:22.945998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af41259a-aa47-4f1e-8a4c-7cd555615722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:22.947165 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41259a-aa47-4f1e-8a4c-7cd555615722 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:22.947712 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41259a-aa47-4f1e-8a4c-7cd555615722 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:22.948213 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af41259a-aa47-4f1e-8a4c-7cd555615722 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:22.950720 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af41259a-aa47-4f1e-8a4c-7cd555615722 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:22.953276 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af41259a-aa47-4f1e-8a4c-7cd555615722 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:22.990599 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 607/1205] 10.222.0.16 () {60 vars in 1132 bytes} [Mon May 23 12:22:22 2022] POST /identity/v3/users => generated 251 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:22:25.373365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e53651-59e3-44d8-895c-7e437fbeac5a None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:25.373933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e53651-59e3-44d8-895c-7e437fbeac5a None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:25.373933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e53651-59e3-44d8-895c-7e437fbeac5a None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:25.375654 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 599/1206] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:22:25.381671 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2f047-eeb2-4c9d-a536-20e8e3334eef None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:25.382126 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2f047-eeb2-4c9d-a536-20e8e3334eef None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:25.382252 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2f047-eeb2-4c9d-a536-20e8e3334eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:25.403401 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-0fe2f047-eeb2-4c9d-a536-20e8e3334eef None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:22:25.404998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-0fe2f047-eeb2-4c9d-a536-20e8e3334eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:22:25.488042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 608/1207] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:22:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 12:22:25.515723 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e53651-59e3-44d8-895c-7e437fbeac5a None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:25.655970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e53651-59e3-44d8-895c-7e437fbeac5a None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:25.698490 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a83291-9a87-4dfe-8130-042ac9dc34f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:25.699290 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a83291-9a87-4dfe-8130-042ac9dc34f5 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:25.699577 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a83291-9a87-4dfe-8130-042ac9dc34f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:25.699897 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a83291-9a87-4dfe-8130-042ac9dc34f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:25.711737 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a83291-9a87-4dfe-8130-042ac9dc34f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:22:25.720529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-c5a83291-9a87-4dfe-8130-042ac9dc34f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 12:22:25.720529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:25.720529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:25.720529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:25.720529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:25.720529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:25.720529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:25.720529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:25.720529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:25.720529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:25.720529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:25.720529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:25.720529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:22:25.720529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:22:25.720529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:22:25.720529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:22:25.722752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:22:25.722752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:22:25.722752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:22:25.722752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:22:25.722752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:25.722752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:25.722752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:25.722752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:25.722752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:25.722752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:25.722752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:25.722752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:22:25.722752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:25.722752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:25.722752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:25.722752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:25.724136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:25.724136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:25.724136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:25.724136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:25.724136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:22:25.724136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:22:25.724136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:22:25.724136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:22:25.724136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:22:25.724136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:22:25.724136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 12:22:25.724136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:22:25.727109 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 600/1208] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 12:22:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:22:25.728185 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2f047-eeb2-4c9d-a536-20e8e3334eef None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:25.778438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:25.850897 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2f047-eeb2-4c9d-a536-20e8e3334eef None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:25.890972 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3451d33a-6509-4f20-81c7-9a4e4b7b19df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:25.891650 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451d33a-6509-4f20-81c7-9a4e4b7b19df None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:25.892019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451d33a-6509-4f20-81c7-9a4e4b7b19df None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:25.892340 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451d33a-6509-4f20-81c7-9a4e4b7b19df None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:25.893092 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3451d33a-6509-4f20-81c7-9a4e4b7b19df None admin] RBAC: Adding query filter params (name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:25.893603 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3451d33a-6509-4f20-81c7-9a4e4b7b19df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:25.894824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3451d33a-6509-4f20-81c7-9a4e4b7b19df None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:25.904710 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 609/1209] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 12:22:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:22:25.911264 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a83291-9a87-4dfe-8130-042ac9dc34f5 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:25.971534 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:26.028876 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a83291-9a87-4dfe-8130-042ac9dc34f5 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:26.073043 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5b7ed-f418-4898-8aee-0ee3c1fe70fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:26.073944 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5b7ed-f418-4898-8aee-0ee3c1fe70fa None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:26.074958 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5b7ed-f418-4898-8aee-0ee3c1fe70fa None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:26.074958 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5b7ed-f418-4898-8aee-0ee3c1fe70fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:26.092278 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e5b7ed-f418-4898-8aee-0ee3c1fe70fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:26.093975 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e5b7ed-f418-4898-8aee-0ee3c1fe70fa None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:26.114090 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-11e5b7ed-f418-4898-8aee-0ee3c1fe70fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:22:26.114090 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:26.114090 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:26.114090 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:26.114090 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:26.114090 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:26.114090 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:26.114090 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:26.114090 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:26.114090 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:26.114090 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:26.114090 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:26.114090 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:22:26.114090 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:22:26.114090 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:22:26.116635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:22:26.116635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:26.116635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:26.116635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:26.116635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:26.116635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:26.116635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:26.116635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:26.116635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:22:26.116635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:26.116635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:26.116635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:26.116635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:26.116635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:26.116635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:26.116635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:26.116635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:26.117673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:22:26.117673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:22:26.117673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:22:26.117673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:22:26.117673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 601/1210] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:22:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:22:26.121282 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3451d33a-6509-4f20-81c7-9a4e4b7b19df None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:26.233693 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:26.236807 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3451d33a-6509-4f20-81c7-9a4e4b7b19df None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:26.278348 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c2bd7-be3b-4745-9b20-cae5383ad210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:26.279375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c2bd7-be3b-4745-9b20-cae5383ad210 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:26.279684 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c2bd7-be3b-4745-9b20-cae5383ad210 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:26.280028 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c2bd7-be3b-4745-9b20-cae5383ad210 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:26.280811 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322c2bd7-be3b-4745-9b20-cae5383ad210 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:26.281157 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322c2bd7-be3b-4745-9b20-cae5383ad210 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:26.282806 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322c2bd7-be3b-4745-9b20-cae5383ad210 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:26.296608 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 610/1211] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:22:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:22:26.302659 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5b7ed-f418-4898-8aee-0ee3c1fe70fa None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:26.414365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5b7ed-f418-4898-8aee-0ee3c1fe70fa None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:26.415950 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:26.454616 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7155cbfc-d324-4353-8001-39a6d067cd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:26.456395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155cbfc-d324-4353-8001-39a6d067cd33 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:26.456846 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155cbfc-d324-4353-8001-39a6d067cd33 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:26.457649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155cbfc-d324-4353-8001-39a6d067cd33 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:26.466734 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7155cbfc-d324-4353-8001-39a6d067cd33 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:26.468436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7155cbfc-d324-4353-8001-39a6d067cd33 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:26.483465 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-7155cbfc-d324-4353-8001-39a6d067cd33 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 12:22:26.483465 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:26.483465 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:26.483465 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:26.483465 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:26.483465 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:26.483465 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:26.483465 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:26.483465 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:26.483465 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:26.483465 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:26.483465 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:26.483465 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:22:26.483465 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:22:26.483465 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:22:26.483465 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:22:26.485205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:26.485205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:26.485205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:22:26.485205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:22:26.485205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:22:26.485205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:22:26.485205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:26.485205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:26.485205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:26.485205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:26.485205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:26.485205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:22:26.485205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:26.485205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:26.485205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:26.485205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:26.485205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:26.485205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:26.487892 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:26.487892 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:26.487892 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:22:26.487892 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:22:26.487892 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:22:26.487892 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:22:26.487892 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:22:26.487892 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:22:26.487892 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 12:22:26.487892 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:22:26.487892 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 602/1212] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:22:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:22:26.491281 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c2bd7-be3b-4745-9b20-cae5383ad210 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:26.609316 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c2bd7-be3b-4745-9b20-cae5383ad210 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:26.653014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14975c86-4927-425a-8a10-fe751492977e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:26.653753 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14975c86-4927-425a-8a10-fe751492977e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:26.654043 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14975c86-4927-425a-8a10-fe751492977e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:26.654376 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14975c86-4927-425a-8a10-fe751492977e None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:26.655492 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14975c86-4927-425a-8a10-fe751492977e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:26.655867 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14975c86-4927-425a-8a10-fe751492977e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:26.657971 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14975c86-4927-425a-8a10-fe751492977e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:26.685732 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 611/1213] 10.222.0.16 () {56 vars in 1143 bytes} [Mon May 23 12:22:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:22:26.687049 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7155cbfc-d324-4353-8001-39a6d067cd33 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:26.825800 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7155cbfc-d324-4353-8001-39a6d067cd33 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:26.867954 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e893124-99c1-408c-b706-f94545e1181b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:26.869258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e893124-99c1-408c-b706-f94545e1181b None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:26.869258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e893124-99c1-408c-b706-f94545e1181b None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:26.869597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e893124-99c1-408c-b706-f94545e1181b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:26.913731 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:26.960954 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e893124-99c1-408c-b706-f94545e1181b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:26.962649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e893124-99c1-408c-b706-f94545e1181b None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:27.085711 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 603/1214] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:22:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 12:22:27.090617 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14975c86-4927-425a-8a10-fe751492977e None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:27.114106 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:27.209377 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14975c86-4927-425a-8a10-fe751492977e None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:27.245708 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf99aef8-1764-4f1b-b44e-b7b15bc36ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:27.246593 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99aef8-1764-4f1b-b44e-b7b15bc36ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:27.246886 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99aef8-1764-4f1b-b44e-b7b15bc36ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:27.247196 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99aef8-1764-4f1b-b44e-b7b15bc36ab8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:27.260098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf99aef8-1764-4f1b-b44e-b7b15bc36ab8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:27.261698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf99aef8-1764-4f1b-b44e-b7b15bc36ab8 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:27.282030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-cf99aef8-1764-4f1b-b44e-b7b15bc36ab8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:22:27.282030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:27.282030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:27.282030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:27.282030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:27.282030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:27.282030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:27.282030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:27.282030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:27.282030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:27.282030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:27.282030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:27.282030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:22:27.282030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:22:27.282030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:22:27.283470 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:22:27.283470 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:27.283470 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:27.283470 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:27.283470 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:27.283470 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:27.283470 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:27.283470 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:27.283470 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:22:27.283470 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:27.283470 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:27.283470 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:27.283470 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:27.283470 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:27.283470 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:27.283470 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:27.283470 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:27.284440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:22:27.284440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:22:27.284440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:22:27.284440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:22:27.284440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 612/1215] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:22:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:22:27.291832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e893124-99c1-408c-b706-f94545e1181b None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:27.354547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:27.395181 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e893124-99c1-408c-b706-f94545e1181b None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:27.430958 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74286ca6-0c98-432d-9e34-16247ee130a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:27.431734 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74286ca6-0c98-432d-9e34-16247ee130a6 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:27.432090 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74286ca6-0c98-432d-9e34-16247ee130a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:27.432462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74286ca6-0c98-432d-9e34-16247ee130a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:27.433273 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74286ca6-0c98-432d-9e34-16247ee130a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:27.433658 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74286ca6-0c98-432d-9e34-16247ee130a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:27.435127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74286ca6-0c98-432d-9e34-16247ee130a6 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:27.452061 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 604/1216] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:22:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:27.453927 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf99aef8-1764-4f1b-b44e-b7b15bc36ab8 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:27.534613 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:27.578683 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf99aef8-1764-4f1b-b44e-b7b15bc36ab8 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:27.622300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371094b7-b9ae-4b3b-9f38-2d95fefe8401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:27.622300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371094b7-b9ae-4b3b-9f38-2d95fefe8401 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:27.622616 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371094b7-b9ae-4b3b-9f38-2d95fefe8401 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:27.622829 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371094b7-b9ae-4b3b-9f38-2d95fefe8401 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:27.631123 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371094b7-b9ae-4b3b-9f38-2d95fefe8401 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:27.632630 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371094b7-b9ae-4b3b-9f38-2d95fefe8401 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:27.647344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-371094b7-b9ae-4b3b-9f38-2d95fefe8401 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:22:27.647344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:27.647344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:27.647344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:27.647344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:27.647344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:27.647344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:27.647344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:27.647344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:27.647344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:27.647344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:27.647344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:27.647344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:22:27.647344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:22:27.647344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:22:27.648543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:22:27.648543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:27.648543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:27.648543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:27.648543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:27.648543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:27.648543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:27.648543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:27.648543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:22:27.648543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:27.648543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:27.648543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:27.648543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:27.648543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:27.648543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:27.648543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:27.648543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:27.654269 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:22:27.654269 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:22:27.654269 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:22:27.654269 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:22:27.654269 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:22:27.654269 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:22:27.654269 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:22:27.654269 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:22:27.654269 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 613/1217] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:22:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:22:27.654782 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74286ca6-0c98-432d-9e34-16247ee130a6 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:27.761798 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74286ca6-0c98-432d-9e34-16247ee130a6 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:27.789023 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:27.806245 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1e136-85e7-4ced-bc3b-cfadb34bb884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:27.807194 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1e136-85e7-4ced-bc3b-cfadb34bb884 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:27.808117 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1e136-85e7-4ced-bc3b-cfadb34bb884 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:27.808839 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1e136-85e7-4ced-bc3b-cfadb34bb884 None admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:27.809505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1e136-85e7-4ced-bc3b-cfadb34bb884 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:27.809850 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1e136-85e7-4ced-bc3b-cfadb34bb884 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:27.813779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1e136-85e7-4ced-bc3b-cfadb34bb884 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:27.828396 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 605/1218] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:22:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:27.835503 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371094b7-b9ae-4b3b-9f38-2d95fefe8401 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:27.949757 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371094b7-b9ae-4b3b-9f38-2d95fefe8401 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:27.983146 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:27.990939 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6b1b2-e003-407f-92cf-a9a118b52280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:27.991765 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6b1b2-e003-407f-92cf-a9a118b52280 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:27.992134 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6b1b2-e003-407f-92cf-a9a118b52280 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:27.992537 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6b1b2-e003-407f-92cf-a9a118b52280 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:27.993459 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6b1b2-e003-407f-92cf-a9a118b52280 None admin] RBAC: Adding query filter params (role.id=81d82471e2be4f0aa8164b75c16a9718, scope.project.id=839585501f414e1ab02501422510e54f, user.id=ebc0f137e0d24dd2a460689bc17a1b98) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:27.993881 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6b1b2-e003-407f-92cf-a9a118b52280 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:27.995388 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6b1b2-e003-407f-92cf-a9a118b52280 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:28.008276 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 614/1219] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:22:27 2022] GET /identity/v3/role_assignments?user.id=ebc0f137e0d24dd2a460689bc17a1b98&scope.project.id=839585501f414e1ab02501422510e54f&role.id=81d82471e2be4f0aa8164b75c16a9718 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:30.249728 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf66b38-45d1-4e47-98a1-b63b365a2633 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:30.250305 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf66b38-45d1-4e47-98a1-b63b365a2633 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:30.250305 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf66b38-45d1-4e47-98a1-b63b365a2633 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:30.252049 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 606/1220] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:22:30.257887 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b54f8-e27c-47ef-82f7-bc5b29bf4ea6 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:30.258456 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b54f8-e27c-47ef-82f7-bc5b29bf4ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:30.258456 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b54f8-e27c-47ef-82f7-bc5b29bf4ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:30.276956 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-412b54f8-e27c-47ef-82f7-bc5b29bf4ea6 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:22:30.278208 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-412b54f8-e27c-47ef-82f7-bc5b29bf4ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:22:30.354260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 615/1221] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:22:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:22:30.377743 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf66b38-45d1-4e47-98a1-b63b365a2633 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:30.488682 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf66b38-45d1-4e47-98a1-b63b365a2633 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:30.525086 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f61cf-e808-409b-9ae6-51f4d6838b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:30.525876 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f61cf-e808-409b-9ae6-51f4d6838b57 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:30.526160 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f61cf-e808-409b-9ae6-51f4d6838b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:30.526436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f61cf-e808-409b-9ae6-51f4d6838b57 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:30.537112 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f61cf-e808-409b-9ae6-51f4d6838b57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:22:30.544685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-471f61cf-e808-409b-9ae6-51f4d6838b57 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 12:22:30.544685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:30.544685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:30.544685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:30.544685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:30.544685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:30.544685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:30.544685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:30.544685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:30.544685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:30.544685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:30.544685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:30.544685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:22:30.544685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:22:30.544685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:22:30.544685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:22:30.545635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:22:30.545635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:22:30.545635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:22:30.545635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:22:30.545635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:30.545635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:30.545635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:30.545635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:30.545635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:30.545635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:30.545635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:30.545635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:22:30.545635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:30.545635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:30.545635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:30.545635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:30.546521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:30.546521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:30.546521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:30.546521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:30.546521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:22:30.546521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:22:30.546521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:22:30.546521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:22:30.546521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:22:30.546521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:22:30.546521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 12:22:30.546521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:22:30.546521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 607/1222] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 12:22:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:22:30.552452 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b54f8-e27c-47ef-82f7-bc5b29bf4ea6 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:30.659658 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b54f8-e27c-47ef-82f7-bc5b29bf4ea6 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:30.685237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:30.696608 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736625a6-1020-4c01-b748-b96779c70f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:30.697284 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736625a6-1020-4c01-b748-b96779c70f6e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:30.697558 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736625a6-1020-4c01-b748-b96779c70f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:30.697858 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736625a6-1020-4c01-b748-b96779c70f6e None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:30.698589 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736625a6-1020-4c01-b748-b96779c70f6e None admin] RBAC: Adding query filter params (name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:30.698906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736625a6-1020-4c01-b748-b96779c70f6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:30.700288 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736625a6-1020-4c01-b748-b96779c70f6e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:30.711229 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 616/1223] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 12:22:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:30.716933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f61cf-e808-409b-9ae6-51f4d6838b57 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:30.767009 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:30.830609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f61cf-e808-409b-9ae6-51f4d6838b57 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:30.868402 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ead41-e482-415b-aa29-040bd8fbca27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:30.869216 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ead41-e482-415b-aa29-040bd8fbca27 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:30.869530 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ead41-e482-415b-aa29-040bd8fbca27 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:30.869810 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ead41-e482-415b-aa29-040bd8fbca27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:30.881866 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ead41-e482-415b-aa29-040bd8fbca27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:30.883528 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ead41-e482-415b-aa29-040bd8fbca27 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:30.905303 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-9b5ead41-e482-415b-aa29-040bd8fbca27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:22:30.905303 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:30.905303 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:30.905303 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:30.905303 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:30.905303 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:30.905303 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:30.905303 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:30.905303 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:30.905303 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:30.905303 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:30.905303 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:30.905303 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:22:30.905303 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:22:30.905303 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:22:30.906282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:22:30.906282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:30.906282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:30.906282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:30.906282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:30.906282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:30.906282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:30.906282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:30.906282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:22:30.906282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:30.906282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:30.906282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:30.906282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:30.906282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:30.906282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:30.906282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:30.906282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:30.907191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:22:30.907191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:22:30.907191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:22:30.907191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:22:30.908459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 608/1224] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:22:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:22:30.912444 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736625a6-1020-4c01-b748-b96779c70f6e None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:31.035135 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736625a6-1020-4c01-b748-b96779c70f6e None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:31.075803 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc7c50a-2f06-4ae0-bda7-7f5acc4ca5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:31.076612 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7c50a-2f06-4ae0-bda7-7f5acc4ca5bb None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:31.076958 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7c50a-2f06-4ae0-bda7-7f5acc4ca5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:31.077300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7c50a-2f06-4ae0-bda7-7f5acc4ca5bb None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:31.078115 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc7c50a-2f06-4ae0-bda7-7f5acc4ca5bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:31.078473 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc7c50a-2f06-4ae0-bda7-7f5acc4ca5bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:31.080152 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc7c50a-2f06-4ae0-bda7-7f5acc4ca5bb None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:31.092996 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 617/1225] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:22:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:31.098961 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ead41-e482-415b-aa29-040bd8fbca27 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:31.209028 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:31.215917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ead41-e482-415b-aa29-040bd8fbca27 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:31.261776 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16842df1-25b9-4093-9535-f9a30f9a8176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:31.262642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16842df1-25b9-4093-9535-f9a30f9a8176 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:31.262928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16842df1-25b9-4093-9535-f9a30f9a8176 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:31.263205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16842df1-25b9-4093-9535-f9a30f9a8176 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:31.272395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16842df1-25b9-4093-9535-f9a30f9a8176 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:31.273778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16842df1-25b9-4093-9535-f9a30f9a8176 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:31.289539 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-16842df1-25b9-4093-9535-f9a30f9a8176 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 12:22:31.289539 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:31.289539 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:31.289539 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:31.289539 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:31.289539 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:31.289539 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:31.289539 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:31.289539 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:31.289539 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:31.289539 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:31.289539 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:31.289539 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:22:31.289539 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:22:31.289539 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:22:31.289539 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:22:31.290704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:31.290704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:31.290704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:22:31.290704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:22:31.290704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:22:31.290704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:22:31.290704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:31.290704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:31.290704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:31.290704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:31.290704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:31.290704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:22:31.290704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:31.290704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:31.290704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:31.290704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:31.290704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:31.290704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:31.292393 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:31.292393 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:31.292393 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:22:31.292393 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:22:31.292393 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:22:31.292393 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:22:31.292393 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:22:31.292393 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:22:31.292393 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 12:22:31.292393 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:22:31.292393 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 609/1226] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:22:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:22:31.296697 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc7c50a-2f06-4ae0-bda7-7f5acc4ca5bb None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:31.346421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:31.402693 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc7c50a-2f06-4ae0-bda7-7f5acc4ca5bb None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:31.441162 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096bcf8-108a-43c4-8d35-8e20e5fb1fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:31.441828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096bcf8-108a-43c4-8d35-8e20e5fb1fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:31.442107 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096bcf8-108a-43c4-8d35-8e20e5fb1fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:31.442384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096bcf8-108a-43c4-8d35-8e20e5fb1fb5 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:31.443207 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c096bcf8-108a-43c4-8d35-8e20e5fb1fb5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:31.443766 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c096bcf8-108a-43c4-8d35-8e20e5fb1fb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:31.445222 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c096bcf8-108a-43c4-8d35-8e20e5fb1fb5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:31.467345 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 618/1227] 10.222.0.16 () {56 vars in 1143 bytes} [Mon May 23 12:22:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:31.473152 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16842df1-25b9-4093-9535-f9a30f9a8176 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:31.590781 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16842df1-25b9-4093-9535-f9a30f9a8176 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:31.632269 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af710b-5693-4d7b-af6d-ebfc19298042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:31.633069 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af710b-5693-4d7b-af6d-ebfc19298042 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:31.633379 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af710b-5693-4d7b-af6d-ebfc19298042 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:31.633742 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af710b-5693-4d7b-af6d-ebfc19298042 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:31.646107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1af710b-5693-4d7b-af6d-ebfc19298042 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:31.647580 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1af710b-5693-4d7b-af6d-ebfc19298042 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:31.664815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-f1af710b-5693-4d7b-af6d-ebfc19298042 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:22:31.664815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:31.664815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:31.664815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:31.664815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:31.664815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:31.664815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:31.664815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:31.664815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:31.664815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:31.664815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:31.664815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:31.664815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:22:31.664815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:22:31.664815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:22:31.665976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:22:31.665976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:31.665976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:31.665976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:31.665976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:31.665976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:31.665976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:31.665976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:31.665976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:22:31.665976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:31.665976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:31.665976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:31.665976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:31.665976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:31.665976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:31.665976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:31.665976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:31.667639 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:22:31.667639 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:22:31.667639 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:22:31.667639 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:22:31.667639 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 610/1228] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:22:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:22:31.672526 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096bcf8-108a-43c4-8d35-8e20e5fb1fb5 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:31.779865 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096bcf8-108a-43c4-8d35-8e20e5fb1fb5 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:31.783544 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:31.821391 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48baa963-8ba0-460f-be67-41cf2a094cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:31.821972 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48baa963-8ba0-460f-be67-41cf2a094cdc None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:31.822253 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48baa963-8ba0-460f-be67-41cf2a094cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:31.822526 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48baa963-8ba0-460f-be67-41cf2a094cdc None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:31.823249 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48baa963-8ba0-460f-be67-41cf2a094cdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:31.823556 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48baa963-8ba0-460f-be67-41cf2a094cdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:31.824947 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48baa963-8ba0-460f-be67-41cf2a094cdc None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:31.837732 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 619/1229] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:22:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:31.843486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af710b-5693-4d7b-af6d-ebfc19298042 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:31.853512 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:31.957842 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af710b-5693-4d7b-af6d-ebfc19298042 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:31.998928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967a5fb-cb92-482e-8808-2bba8a08a281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:31.999723 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967a5fb-cb92-482e-8808-2bba8a08a281 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:32.000002 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967a5fb-cb92-482e-8808-2bba8a08a281 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:32.000302 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967a5fb-cb92-482e-8808-2bba8a08a281 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:32.007551 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967a5fb-cb92-482e-8808-2bba8a08a281 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:32.009064 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967a5fb-cb92-482e-8808-2bba8a08a281 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:32.023558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-6967a5fb-cb92-482e-8808-2bba8a08a281 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:22:32.023558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:32.023558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:32.023558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:32.023558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:32.023558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:32.023558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:32.023558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:32.023558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:32.023558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:32.023558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:32.023558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:32.023558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:22:32.023558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:22:32.023558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:22:32.024755 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:22:32.024755 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:32.024755 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:32.024755 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:32.024755 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:32.024755 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:32.024755 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:32.024755 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:32.024755 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:22:32.024755 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:32.024755 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:32.024755 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:32.024755 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:32.024755 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:32.024755 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:32.024755 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:32.024755 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:32.025848 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:22:32.025848 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:22:32.025848 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:22:32.025848 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:22:32.025848 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:22:32.025848 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:22:32.025848 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:22:32.025848 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:22:32.025848 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 611/1230] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:22:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:22:32.030795 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48baa963-8ba0-460f-be67-41cf2a094cdc None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:32.147429 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48baa963-8ba0-460f-be67-41cf2a094cdc None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:32.182537 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee80b9b9-9696-4599-9ed6-338333079883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:32.183200 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80b9b9-9696-4599-9ed6-338333079883 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:32.183502 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80b9b9-9696-4599-9ed6-338333079883 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:32.183753 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80b9b9-9696-4599-9ed6-338333079883 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:32.184559 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee80b9b9-9696-4599-9ed6-338333079883 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:32.184871 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee80b9b9-9696-4599-9ed6-338333079883 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:32.186287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee80b9b9-9696-4599-9ed6-338333079883 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:32.199915 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 620/1231] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:22:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:32.206529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967a5fb-cb92-482e-8808-2bba8a08a281 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:32.281909 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:32.321276 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967a5fb-cb92-482e-8808-2bba8a08a281 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:32.361471 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e65d58-4926-43ca-a740-bfb951aa4715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:32.362378 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e65d58-4926-43ca-a740-bfb951aa4715 None admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:32.362719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e65d58-4926-43ca-a740-bfb951aa4715 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:32.363054 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e65d58-4926-43ca-a740-bfb951aa4715 None admin] PATH_INFO: `/v3/projects/839585501f414e1ab02501422510e54f/users/ebc0f137e0d24dd2a460689bc17a1b98/roles/81d82471e2be4f0aa8164b75c16a9718` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:32.383984 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e65d58-4926-43ca-a740-bfb951aa4715 None admin] RBAC: Authorizing `identity:create_grant(project_id=839585501f414e1ab02501422510e54f, user_id=ebc0f137e0d24dd2a460689bc17a1b98, role_id=81d82471e2be4f0aa8164b75c16a9718)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:32.385891 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e65d58-4926-43ca-a740-bfb951aa4715 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:32.400190 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 612/1232] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 12:22:32 2022] PUT /identity/v3/projects/839585501f414e1ab02501422510e54f/users/ebc0f137e0d24dd2a460689bc17a1b98/roles/81d82471e2be4f0aa8164b75c16a9718 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:22:34.655181 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d5c63-09a4-4b68-9575-938e1b4646a3 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:34.655740 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d5c63-09a4-4b68-9575-938e1b4646a3 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:34.655740 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d5c63-09a4-4b68-9575-938e1b4646a3 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:34.657137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 621/1233] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:22:34.663256 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8291efb-f581-4c14-9045-cd7d6bc875fb None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:34.663719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8291efb-f581-4c14-9045-cd7d6bc875fb None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:34.663824 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8291efb-f581-4c14-9045-cd7d6bc875fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:34.682952 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-d8291efb-f581-4c14-9045-cd7d6bc875fb None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:22:34.684360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-d8291efb-f581-4c14-9045-cd7d6bc875fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:22:34.756766 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 613/1234] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:22:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:22:34.780840 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d5c63-09a4-4b68-9575-938e1b4646a3 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:34.809275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:34.897499 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d5c63-09a4-4b68-9575-938e1b4646a3 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:34.938636 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2edc2-97fc-4031-a534-d7494dd5aefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:34.939365 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2edc2-97fc-4031-a534-d7494dd5aefb None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:34.939657 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2edc2-97fc-4031-a534-d7494dd5aefb None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:34.940107 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2edc2-97fc-4031-a534-d7494dd5aefb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:34.954677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2edc2-97fc-4031-a534-d7494dd5aefb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:22:34.965408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-93c2edc2-97fc-4031-a534-d7494dd5aefb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 12:22:34.965408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:34.965408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:34.965408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:34.965408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:34.965408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:34.965408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:34.965408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:34.965408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:34.965408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:34.965408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:34.965408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:34.965408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:22:34.965408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:22:34.965408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:22:34.965408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:22:34.966776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:22:34.966776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:22:34.966776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:22:34.966776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:22:34.966776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:34.966776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:34.966776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:34.966776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:34.966776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:34.966776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:34.966776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:34.966776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:22:34.966776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:34.966776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:34.966776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:34.966776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:34.967987 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:34.967987 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:34.967987 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:34.967987 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:34.967987 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:22:34.967987 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:22:34.967987 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:22:34.967987 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:22:34.967987 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:22:34.967987 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:22:34.967987 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 12:22:34.967987 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:22:34.967987 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 622/1235] 10.222.0.16 () {56 vars in 1102 bytes} [Mon May 23 12:22:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:22:34.974420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8291efb-f581-4c14-9045-cd7d6bc875fb None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:35.025594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:35.092036 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8291efb-f581-4c14-9045-cd7d6bc875fb None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:35.132497 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e531b63-f1f9-446d-b1b9-7f9893bfabcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:35.133334 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e531b63-f1f9-446d-b1b9-7f9893bfabcf None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:35.133646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e531b63-f1f9-446d-b1b9-7f9893bfabcf None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:35.133995 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e531b63-f1f9-446d-b1b9-7f9893bfabcf None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:35.134978 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e531b63-f1f9-446d-b1b9-7f9893bfabcf None admin] RBAC: Adding query filter params (name=service) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:35.135294 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e531b63-f1f9-446d-b1b9-7f9893bfabcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:35.136719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e531b63-f1f9-446d-b1b9-7f9893bfabcf None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:35.145373 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 614/1236] 10.222.0.16 () {56 vars in 1103 bytes} [Mon May 23 12:22:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:35.151031 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2edc2-97fc-4031-a534-d7494dd5aefb None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:35.257127 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:35.277722 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2edc2-97fc-4031-a534-d7494dd5aefb None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:35.323284 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc3c7b-06f3-4438-8cbb-dc4ff8076714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:35.324089 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc3c7b-06f3-4438-8cbb-dc4ff8076714 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:35.324409 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc3c7b-06f3-4438-8cbb-dc4ff8076714 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:35.324690 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc3c7b-06f3-4438-8cbb-dc4ff8076714 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:35.339970 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc3c7b-06f3-4438-8cbb-dc4ff8076714 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:35.342311 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc3c7b-06f3-4438-8cbb-dc4ff8076714 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:35.365525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-dccc3c7b-06f3-4438-8cbb-dc4ff8076714 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:22:35.365525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:35.365525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:35.365525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:35.365525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:35.365525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:35.365525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:35.365525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:35.365525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:35.365525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:35.365525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:35.365525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:35.365525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:22:35.365525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:22:35.365525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:22:35.366615 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:22:35.366615 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:35.366615 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:35.366615 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:35.366615 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:35.366615 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:35.366615 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:35.366615 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:35.366615 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:22:35.366615 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:35.366615 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:35.366615 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:35.366615 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:35.366615 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:35.366615 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:35.366615 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:35.366615 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:35.367595 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:22:35.367595 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:22:35.367595 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:22:35.367595 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:22:35.367595 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 623/1237] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:22:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:22:35.373036 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e531b63-f1f9-446d-b1b9-7f9893bfabcf None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:35.488600 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e531b63-f1f9-446d-b1b9-7f9893bfabcf None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:35.498548 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:35.527572 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b107b-b57e-4cba-a711-cc4df91e6b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:35.528184 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b107b-b57e-4cba-a711-cc4df91e6b2f None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:35.528495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b107b-b57e-4cba-a711-cc4df91e6b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:35.528806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b107b-b57e-4cba-a711-cc4df91e6b2f None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:35.529541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b107b-b57e-4cba-a711-cc4df91e6b2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:35.529839 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b107b-b57e-4cba-a711-cc4df91e6b2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:35.531209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b107b-b57e-4cba-a711-cc4df91e6b2f None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:35.543198 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 615/1238] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:22:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:35.549752 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc3c7b-06f3-4438-8cbb-dc4ff8076714 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:35.667120 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc3c7b-06f3-4438-8cbb-dc4ff8076714 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:35.704957 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7261f-b61f-44ea-bef0-5b79cd0a1226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:35.705773 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7261f-b61f-44ea-bef0-5b79cd0a1226 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:35.706064 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7261f-b61f-44ea-bef0-5b79cd0a1226 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:35.706434 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7261f-b61f-44ea-bef0-5b79cd0a1226 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:35.709932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:35.717128 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7261f-b61f-44ea-bef0-5b79cd0a1226 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:35.718834 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7261f-b61f-44ea-bef0-5b79cd0a1226 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:35.735542 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-57b7261f-b61f-44ea-bef0-5b79cd0a1226 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 12:22:35.735542 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:35.735542 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:35.735542 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:35.735542 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:35.735542 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:35.735542 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:35.735542 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:35.735542 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:35.735542 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:35.735542 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:35.735542 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:35.735542 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:22:35.735542 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:22:35.735542 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:22:35.735542 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:22:35.737080 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:35.737080 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:35.737080 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:22:35.737080 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:22:35.737080 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:22:35.737080 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:22:35.737080 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:35.737080 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:35.737080 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:35.737080 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:35.737080 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:35.737080 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:22:35.737080 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:35.737080 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:35.737080 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:35.737080 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:35.737080 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:35.737080 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:35.738124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:35.738124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:35.738124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:22:35.738124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:22:35.738124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:22:35.738124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:22:35.738124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:22:35.738124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:22:35.738124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 12:22:35.738124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:22:35.738124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 624/1239] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:22:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:22:35.743381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b107b-b57e-4cba-a711-cc4df91e6b2f None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:35.859547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b107b-b57e-4cba-a711-cc4df91e6b2f None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:35.896914 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b4be7-0f58-4a1f-ae96-ea3edc159624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:35.897669 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b4be7-0f58-4a1f-ae96-ea3edc159624 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:35.898001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b4be7-0f58-4a1f-ae96-ea3edc159624 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:35.898345 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b4be7-0f58-4a1f-ae96-ea3edc159624 None admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:35.899301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4b4be7-0f58-4a1f-ae96-ea3edc159624 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:35.899632 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4b4be7-0f58-4a1f-ae96-ea3edc159624 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:35.901376 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4b4be7-0f58-4a1f-ae96-ea3edc159624 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:35.920472 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 616/1240] 10.222.0.16 () {56 vars in 1143 bytes} [Mon May 23 12:22:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:35.927271 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7261f-b61f-44ea-bef0-5b79cd0a1226 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:36.044505 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7261f-b61f-44ea-bef0-5b79cd0a1226 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:36.082171 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb047ab-be46-4dad-a744-9e6a0a2d36c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:36.082854 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb047ab-be46-4dad-a744-9e6a0a2d36c8 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:36.083149 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb047ab-be46-4dad-a744-9e6a0a2d36c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:36.083462 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb047ab-be46-4dad-a744-9e6a0a2d36c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:36.151688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:36.159703 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb047ab-be46-4dad-a744-9e6a0a2d36c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:36.161328 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb047ab-be46-4dad-a744-9e6a0a2d36c8 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:36.277219 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 625/1241] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:22:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 12:22:36.284441 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b4be7-0f58-4a1f-ae96-ea3edc159624 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:36.320547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:36.410925 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b4be7-0f58-4a1f-ae96-ea3edc159624 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:36.450280 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2defe-6c69-4891-b146-4114aa7d49a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:36.451072 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2defe-6c69-4891-b146-4114aa7d49a4 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:36.451356 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2defe-6c69-4891-b146-4114aa7d49a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:36.451635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2defe-6c69-4891-b146-4114aa7d49a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:36.462881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2defe-6c69-4891-b146-4114aa7d49a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:36.464383 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2defe-6c69-4891-b146-4114aa7d49a4 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:36.484191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-bfd2defe-6c69-4891-b146-4114aa7d49a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:22:36.484191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:36.484191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:36.484191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:36.484191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:36.484191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:36.484191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:36.484191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:36.484191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:36.484191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:36.484191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:36.484191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:36.484191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:22:36.484191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:22:36.484191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:22:36.485292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:22:36.485292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:36.485292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:36.485292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:36.485292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:36.485292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:36.485292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:36.485292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:36.485292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:22:36.485292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:36.485292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:36.485292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:36.485292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:36.485292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:36.485292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:36.485292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:36.485292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:36.488149 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:22:36.488149 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:22:36.488149 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:22:36.488149 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:22:36.488149 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 617/1242] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:22:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:22:36.492284 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb047ab-be46-4dad-a744-9e6a0a2d36c8 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:36.601057 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:36.608730 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb047ab-be46-4dad-a744-9e6a0a2d36c8 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:36.647172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f8e0f-c177-47cd-a8fd-d5a45e6faf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:36.647840 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f8e0f-c177-47cd-a8fd-d5a45e6faf9e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:36.648125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f8e0f-c177-47cd-a8fd-d5a45e6faf9e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:36.648438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f8e0f-c177-47cd-a8fd-d5a45e6faf9e None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:36.649165 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f8e0f-c177-47cd-a8fd-d5a45e6faf9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:36.649480 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f8e0f-c177-47cd-a8fd-d5a45e6faf9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:36.650861 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f8e0f-c177-47cd-a8fd-d5a45e6faf9e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:36.663805 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 626/1243] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:22:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:36.669414 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2defe-6c69-4891-b146-4114aa7d49a4 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:36.755469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:36.779195 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2defe-6c69-4891-b146-4114aa7d49a4 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:36.815356 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346e089-e643-4439-9e6e-c84bfd40cd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:36.816159 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346e089-e643-4439-9e6e-c84bfd40cd71 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:36.816464 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346e089-e643-4439-9e6e-c84bfd40cd71 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:36.816781 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5346e089-e643-4439-9e6e-c84bfd40cd71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:36.824206 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5346e089-e643-4439-9e6e-c84bfd40cd71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:36.825670 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5346e089-e643-4439-9e6e-c84bfd40cd71 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:36.839808 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-5346e089-e643-4439-9e6e-c84bfd40cd71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:22:36.839808 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:36.839808 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:36.839808 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:36.839808 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:36.839808 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:36.839808 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:36.839808 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:36.839808 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:36.839808 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:36.839808 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:36.839808 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:36.839808 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:22:36.839808 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:22:36.839808 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:22:36.840993 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:22:36.840993 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:36.840993 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:36.840993 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:36.840993 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:36.840993 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:36.840993 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:36.840993 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:36.840993 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:22:36.840993 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:36.840993 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:36.840993 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:36.840993 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:36.840993 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:36.840993 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:36.840993 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:36.840993 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:36.841917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:22:36.841917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:22:36.841917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:22:36.841917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:22:36.841917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:22:36.841917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:22:36.841917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:22:36.841917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:22:36.841917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 618/1244] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:22:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:22:36.847089 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f8e0f-c177-47cd-a8fd-d5a45e6faf9e None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:36.958307 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f8e0f-c177-47cd-a8fd-d5a45e6faf9e None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:36.995084 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e949d33-dde5-4348-85bd-2c28b892c186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:36.995831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e949d33-dde5-4348-85bd-2c28b892c186 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:36.996110 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e949d33-dde5-4348-85bd-2c28b892c186 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:36.996433 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e949d33-dde5-4348-85bd-2c28b892c186 None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:36.997308 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e949d33-dde5-4348-85bd-2c28b892c186 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:36.997624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e949d33-dde5-4348-85bd-2c28b892c186 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:36.999239 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e949d33-dde5-4348-85bd-2c28b892c186 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:37.014450 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 627/1245] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:22:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:37.021693 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346e089-e643-4439-9e6e-c84bfd40cd71 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:37.134788 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5346e089-e643-4439-9e6e-c84bfd40cd71 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:37.169564 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1809a-f0cc-4052-bac4-59f6d44fdfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:37.170222 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1809a-f0cc-4052-bac4-59f6d44fdfe7 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:37.170507 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1809a-f0cc-4052-bac4-59f6d44fdfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:37.170786 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1809a-f0cc-4052-bac4-59f6d44fdfe7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:37.171630 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a1809a-f0cc-4052-bac4-59f6d44fdfe7 None admin] RBAC: Adding query filter params (role.id=81d82471e2be4f0aa8164b75c16a9718, scope.project.id=839585501f414e1ab02501422510e54f, user.id=ebc0f137e0d24dd2a460689bc17a1b98) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:37.171977 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a1809a-f0cc-4052-bac4-59f6d44fdfe7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:37.173639 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a1809a-f0cc-4052-bac4-59f6d44fdfe7 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:37.179969 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 619/1246] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:22:37 2022] GET /identity/v3/role_assignments?user.id=ebc0f137e0d24dd2a460689bc17a1b98&scope.project.id=839585501f414e1ab02501422510e54f&role.id=81d82471e2be4f0aa8164b75c16a9718 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:39.649716 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ebfed7-0b1f-448f-be32-7f68e3b43fde None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:39.650247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ebfed7-0b1f-448f-be32-7f68e3b43fde None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:39.650247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ebfed7-0b1f-448f-be32-7f68e3b43fde None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:39.651623 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 628/1247] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:22:39.656988 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f27efd-9aa9-460f-a413-950cb0cf53ff None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:39.657575 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f27efd-9aa9-460f-a413-950cb0cf53ff None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:39.657575 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f27efd-9aa9-460f-a413-950cb0cf53ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:39.662626 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:39.676551 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-19f27efd-9aa9-460f-a413-950cb0cf53ff None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:22:39.677921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-19f27efd-9aa9-460f-a413-950cb0cf53ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:22:39.755683 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 620/1248] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:22:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:22:39.780027 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ebfed7-0b1f-448f-be32-7f68e3b43fde None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:39.804030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:39.913284 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ebfed7-0b1f-448f-be32-7f68e3b43fde None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:39.951566 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5bf3a0-18e7-4443-9186-da6ce1f31767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:39.952448 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5bf3a0-18e7-4443-9186-da6ce1f31767 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:39.952736 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5bf3a0-18e7-4443-9186-da6ce1f31767 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:39.953018 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5bf3a0-18e7-4443-9186-da6ce1f31767 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:39.963918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5bf3a0-18e7-4443-9186-da6ce1f31767 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:22:39.972453 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-be5bf3a0-18e7-4443-9186-da6ce1f31767 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:22:39.972453 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:39.972453 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:39.972453 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:39.972453 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:39.972453 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:39.972453 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:39.972453 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:39.972453 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:39.972453 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:39.972453 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:39.972453 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:39.972453 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:22:39.972453 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:22:39.972453 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:22:39.972453 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:22:39.973709 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:22:39.973709 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:22:39.973709 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:22:39.973709 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:22:39.973709 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:39.973709 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:39.973709 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:39.973709 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:39.973709 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:39.973709 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:39.973709 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:39.973709 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:22:39.973709 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:39.973709 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:39.973709 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:39.973709 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:39.974753 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:39.974753 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:39.974753 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:39.974753 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:39.974753 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:22:39.974753 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:22:39.974753 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:22:39.974753 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:22:39.974753 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:22:39.974753 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:22:39.974753 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:22:39.974753 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:22:39.974753 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 629/1249] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:22:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:22:39.982121 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f27efd-9aa9-460f-a413-950cb0cf53ff None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:40.094327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f27efd-9aa9-460f-a413-950cb0cf53ff None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:40.132483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:40.138663 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cf815-a2ef-4158-9cde-8f330c2fe9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:40.139583 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cf815-a2ef-4158-9cde-8f330c2fe9f8 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:40.140656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cf815-a2ef-4158-9cde-8f330c2fe9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:40.140947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cf815-a2ef-4158-9cde-8f330c2fe9f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:40.141977 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7cf815-a2ef-4158-9cde-8f330c2fe9f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:40.142466 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7cf815-a2ef-4158-9cde-8f330c2fe9f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:40.144188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7cf815-a2ef-4158-9cde-8f330c2fe9f8 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:40.153638 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 621/1250] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:22:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:40.159989 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5bf3a0-18e7-4443-9186-da6ce1f31767 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:40.249846 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:40.275317 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5bf3a0-18e7-4443-9186-da6ce1f31767 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:40.314266 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ba31c-71cf-4280-9528-4a87d3301126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:40.315031 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ba31c-71cf-4280-9528-4a87d3301126 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:40.315979 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ba31c-71cf-4280-9528-4a87d3301126 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:40.315979 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ba31c-71cf-4280-9528-4a87d3301126 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:40.327241 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ba31c-71cf-4280-9528-4a87d3301126 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:40.328793 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ba31c-71cf-4280-9528-4a87d3301126 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:40.347210 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-3c0ba31c-71cf-4280-9528-4a87d3301126 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:22:40.347210 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:40.347210 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:40.347210 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:40.347210 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:40.347210 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:40.347210 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:40.347210 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:40.347210 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:40.347210 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:40.347210 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:40.347210 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:40.347210 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:22:40.347210 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:22:40.347210 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:22:40.348264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:22:40.348264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:40.348264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:40.348264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:40.348264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:40.348264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:40.348264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:40.348264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:40.348264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:22:40.348264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:40.348264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:40.348264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:40.348264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:40.348264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:40.348264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:40.348264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:40.348264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:40.349212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:22:40.349212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:22:40.349212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:22:40.349212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:22:40.349212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 630/1251] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:22:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:22:40.354356 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cf815-a2ef-4158-9cde-8f330c2fe9f8 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:40.462246 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cf815-a2ef-4158-9cde-8f330c2fe9f8 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:40.503542 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d9787-46d3-4797-bd90-bae2f699e076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:40.504216 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d9787-46d3-4797-bd90-bae2f699e076 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:40.504504 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d9787-46d3-4797-bd90-bae2f699e076 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:40.504833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d9787-46d3-4797-bd90-bae2f699e076 None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:40.505829 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d9787-46d3-4797-bd90-bae2f699e076 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:40.506268 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d9787-46d3-4797-bd90-bae2f699e076 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:40.508381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d9787-46d3-4797-bd90-bae2f699e076 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:40.524293 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 622/1252] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:22:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:22:40.530419 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ba31c-71cf-4280-9528-4a87d3301126 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:40.640515 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ba31c-71cf-4280-9528-4a87d3301126 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:40.670524 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:40.680970 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb771b-eb06-4f59-a29c-f315a75b9d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:40.681838 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb771b-eb06-4f59-a29c-f315a75b9d60 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:40.682164 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb771b-eb06-4f59-a29c-f315a75b9d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:40.682482 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb771b-eb06-4f59-a29c-f315a75b9d60 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:40.691551 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb771b-eb06-4f59-a29c-f315a75b9d60 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:40.693399 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb771b-eb06-4f59-a29c-f315a75b9d60 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:40.711469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-18eb771b-eb06-4f59-a29c-f315a75b9d60 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 12:22:40.711469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:40.711469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:40.711469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:40.711469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:40.711469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:40.711469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:40.711469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:40.711469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:40.711469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:40.711469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:40.711469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:40.711469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:22:40.711469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:22:40.711469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:22:40.711469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:22:40.712698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:40.712698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:40.712698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:22:40.712698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:22:40.712698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:22:40.712698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:22:40.712698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:40.712698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:40.712698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:40.712698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:40.712698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:40.712698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:22:40.712698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:40.712698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:40.712698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:40.712698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:40.712698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:40.712698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:40.713801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:40.713801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:40.713801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:22:40.713801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:22:40.713801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:22:40.713801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:22:40.713801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:22:40.713801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:22:40.713801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 12:22:40.713801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:22:40.713801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 631/1253] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:22:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:22:40.719786 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d9787-46d3-4797-bd90-bae2f699e076 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:40.781172 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:40.849440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d9787-46d3-4797-bd90-bae2f699e076 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:40.888212 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b51c4-e27d-4a50-9424-0cac17e10e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:40.888893 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b51c4-e27d-4a50-9424-0cac17e10e75 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:40.889179 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b51c4-e27d-4a50-9424-0cac17e10e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:40.889455 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b51c4-e27d-4a50-9424-0cac17e10e75 None admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:40.890713 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b51c4-e27d-4a50-9424-0cac17e10e75 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:40.891300 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b51c4-e27d-4a50-9424-0cac17e10e75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:40.893946 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b51c4-e27d-4a50-9424-0cac17e10e75 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:40.913188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 623/1254] 10.222.0.16 () {56 vars in 1143 bytes} [Mon May 23 12:22:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:40.919533 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb771b-eb06-4f59-a29c-f315a75b9d60 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:41.037299 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb771b-eb06-4f59-a29c-f315a75b9d60 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:41.072964 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6060c87-3cfb-41e8-bcc1-bce92b998e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:41.073622 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6060c87-3cfb-41e8-bcc1-bce92b998e83 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:41.073912 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6060c87-3cfb-41e8-bcc1-bce92b998e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:41.074188 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6060c87-3cfb-41e8-bcc1-bce92b998e83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:41.135378 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:41.149079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6060c87-3cfb-41e8-bcc1-bce92b998e83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:41.150548 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6060c87-3cfb-41e8-bcc1-bce92b998e83 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:41.252676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 632/1255] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:22:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 12:22:41.258803 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b51c4-e27d-4a50-9424-0cac17e10e75 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:41.365927 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b51c4-e27d-4a50-9424-0cac17e10e75 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:41.381102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:41.403851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b66e35-0197-4a21-932d-48d938651b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:41.404671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b66e35-0197-4a21-932d-48d938651b8d None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:41.404999 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b66e35-0197-4a21-932d-48d938651b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:41.405308 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b66e35-0197-4a21-932d-48d938651b8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:41.416454 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b66e35-0197-4a21-932d-48d938651b8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:41.418137 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b66e35-0197-4a21-932d-48d938651b8d None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:41.437722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-a0b66e35-0197-4a21-932d-48d938651b8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:22:41.437722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:41.437722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:41.437722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:41.437722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:41.437722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:41.437722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:41.437722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:41.437722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:41.437722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:41.437722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:41.437722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:41.437722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:22:41.437722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:22:41.437722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:22:41.438742 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:22:41.438742 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:41.438742 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:41.438742 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:41.438742 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:41.438742 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:41.438742 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:41.438742 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:41.438742 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:22:41.438742 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:41.438742 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:41.438742 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:41.438742 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:41.438742 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:41.438742 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:41.438742 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:41.438742 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:41.439692 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:22:41.439692 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:22:41.439692 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:22:41.439692 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:22:41.439692 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 624/1256] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:22:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 12:22:41.444960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6060c87-3cfb-41e8-bcc1-bce92b998e83 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:41.546444 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:41.559433 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6060c87-3cfb-41e8-bcc1-bce92b998e83 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:41.598814 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e5fb3b-b06b-43c9-a3e8-3ab8c1da38d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:41.599578 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5fb3b-b06b-43c9-a3e8-3ab8c1da38d8 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:41.599899 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5fb3b-b06b-43c9-a3e8-3ab8c1da38d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:41.600193 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5fb3b-b06b-43c9-a3e8-3ab8c1da38d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:41.600932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e5fb3b-b06b-43c9-a3e8-3ab8c1da38d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:41.601243 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e5fb3b-b06b-43c9-a3e8-3ab8c1da38d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:41.602615 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e5fb3b-b06b-43c9-a3e8-3ab8c1da38d8 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:41.615008 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 633/1257] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:22:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:41.620714 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b66e35-0197-4a21-932d-48d938651b8d None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:41.743742 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b66e35-0197-4a21-932d-48d938651b8d None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:41.780800 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f4817-695d-4320-a8cc-f4f13336170c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:41.781621 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f4817-695d-4320-a8cc-f4f13336170c None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:41.781910 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f4817-695d-4320-a8cc-f4f13336170c None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:41.782260 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f4817-695d-4320-a8cc-f4f13336170c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:41.791127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f4817-695d-4320-a8cc-f4f13336170c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:41.792716 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f4817-695d-4320-a8cc-f4f13336170c None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:41.809161 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-124f4817-695d-4320-a8cc-f4f13336170c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:22:41.809161 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:41.809161 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:41.809161 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:41.809161 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:41.809161 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:41.809161 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:41.809161 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:41.809161 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:41.809161 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:41.809161 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:41.809161 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:41.809161 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:22:41.809161 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:22:41.809161 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:22:41.810572 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:22:41.810572 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:41.810572 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:41.810572 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:41.810572 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:41.810572 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:41.810572 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:41.810572 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:41.810572 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:22:41.810572 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:41.810572 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:41.810572 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:41.810572 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:41.810572 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:41.810572 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:41.810572 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:41.810572 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:41.811817 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:22:41.811817 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:22:41.811817 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:22:41.811817 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:22:41.811817 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:22:41.811817 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:22:41.811817 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:22:41.811817 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:22:41.816411 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 625/1258] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:22:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:22:41.817437 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e5fb3b-b06b-43c9-a3e8-3ab8c1da38d8 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:41.926367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e5fb3b-b06b-43c9-a3e8-3ab8c1da38d8 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:41.961738 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656998d-5aff-4b06-a7a8-0395009f660b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:41.962414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656998d-5aff-4b06-a7a8-0395009f660b None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:41.962701 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656998d-5aff-4b06-a7a8-0395009f660b None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:41.962981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656998d-5aff-4b06-a7a8-0395009f660b None admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:41.963779 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656998d-5aff-4b06-a7a8-0395009f660b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:41.964095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656998d-5aff-4b06-a7a8-0395009f660b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:41.965608 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656998d-5aff-4b06-a7a8-0395009f660b None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:41.983819 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 634/1259] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:22:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:41.985064 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f4817-695d-4320-a8cc-f4f13336170c None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:42.014054 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:42.096762 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f4817-695d-4320-a8cc-f4f13336170c None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:42.132675 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960330cf-5641-4f7f-92a4-f91696defdc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:42.133348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960330cf-5641-4f7f-92a4-f91696defdc3 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:42.133638 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960330cf-5641-4f7f-92a4-f91696defdc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:42.133921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960330cf-5641-4f7f-92a4-f91696defdc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:42.134766 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960330cf-5641-4f7f-92a4-f91696defdc3 None admin] RBAC: Adding query filter params (role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8, scope.project.id=839585501f414e1ab02501422510e54f, user.id=ebc0f137e0d24dd2a460689bc17a1b98) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:42.135085 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960330cf-5641-4f7f-92a4-f91696defdc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:42.136581 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960330cf-5641-4f7f-92a4-f91696defdc3 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:42.148368 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 626/1260] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:22:41 2022] GET /identity/v3/role_assignments?user.id=ebc0f137e0d24dd2a460689bc17a1b98&scope.project.id=839585501f414e1ab02501422510e54f&role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:44.315422 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a853901-885c-422c-a2e4-26b39d7d87f0 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:44.315924 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a853901-885c-422c-a2e4-26b39d7d87f0 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:44.315924 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a853901-885c-422c-a2e4-26b39d7d87f0 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:44.317296 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 635/1261] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:22:44.322694 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edb3a5-f1e5-4ed2-a9b6-839b4526d669 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:44.323129 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edb3a5-f1e5-4ed2-a9b6-839b4526d669 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:44.323261 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edb3a5-f1e5-4ed2-a9b6-839b4526d669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:44.340359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-94edb3a5-f1e5-4ed2-a9b6-839b4526d669 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:22:44.341548 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-94edb3a5-f1e5-4ed2-a9b6-839b4526d669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:22:44.406034 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 627/1262] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:22:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:22:44.429083 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a853901-885c-422c-a2e4-26b39d7d87f0 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:44.440604 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:44.529118 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a853901-885c-422c-a2e4-26b39d7d87f0 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:44.561047 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e68a0-626e-4359-b9af-bbc34d13d89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:44.561823 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e68a0-626e-4359-b9af-bbc34d13d89f None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:44.562103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e68a0-626e-4359-b9af-bbc34d13d89f None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:44.562376 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e68a0-626e-4359-b9af-bbc34d13d89f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:44.573262 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1e68a0-626e-4359-b9af-bbc34d13d89f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:22:44.581676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-de1e68a0-626e-4359-b9af-bbc34d13d89f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:22:44.581676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:44.581676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:44.581676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:44.581676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:44.581676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:44.581676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:44.581676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:44.581676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:44.581676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:44.581676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:44.581676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:44.581676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:22:44.581676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:22:44.581676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:22:44.581676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:22:44.582777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:22:44.582777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:22:44.582777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:22:44.582777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:22:44.582777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:44.582777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:44.582777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:44.582777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:44.582777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:44.582777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:44.582777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:44.582777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:22:44.582777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:44.582777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:44.582777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:44.582777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:44.583788 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:44.583788 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:44.583788 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:44.583788 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:44.583788 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:22:44.583788 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:22:44.583788 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:22:44.583788 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:22:44.583788 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:22:44.583788 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:22:44.583788 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:22:44.583788 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:22:44.583788 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 636/1263] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:22:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:22:44.589020 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94edb3a5-f1e5-4ed2-a9b6-839b4526d669 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:44.607245 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:44.686669 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94edb3a5-f1e5-4ed2-a9b6-839b4526d669 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:44.722060 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd40f3e-a95e-48c5-b6b6-ccae2b90d843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:44.722712 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd40f3e-a95e-48c5-b6b6-ccae2b90d843 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:44.722994 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd40f3e-a95e-48c5-b6b6-ccae2b90d843 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:44.723271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd40f3e-a95e-48c5-b6b6-ccae2b90d843 None admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:44.724010 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd40f3e-a95e-48c5-b6b6-ccae2b90d843 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:44.724348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd40f3e-a95e-48c5-b6b6-ccae2b90d843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:44.725715 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd40f3e-a95e-48c5-b6b6-ccae2b90d843 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:44.733418 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 628/1264] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:22:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:44.739235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e68a0-626e-4359-b9af-bbc34d13d89f None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:44.810419 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:44.850157 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e68a0-626e-4359-b9af-bbc34d13d89f None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:44.884353 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6eef62-ec65-43b5-a8a2-22c9cf955a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:44.885141 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6eef62-ec65-43b5-a8a2-22c9cf955a59 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:44.885427 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6eef62-ec65-43b5-a8a2-22c9cf955a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:44.885705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6eef62-ec65-43b5-a8a2-22c9cf955a59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:44.896552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6eef62-ec65-43b5-a8a2-22c9cf955a59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:44.898011 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6eef62-ec65-43b5-a8a2-22c9cf955a59 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:44.914517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-1e6eef62-ec65-43b5-a8a2-22c9cf955a59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:22:44.914517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:44.914517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:44.914517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:44.914517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:44.914517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:44.914517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:44.914517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:44.914517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:44.914517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:44.914517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:44.914517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:44.914517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:22:44.914517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:22:44.914517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:22:44.915585 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:22:44.915585 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:44.915585 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:44.915585 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:44.915585 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:44.915585 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:44.915585 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:44.915585 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:44.915585 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:22:44.915585 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:44.915585 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:44.915585 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:44.915585 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:44.915585 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:44.915585 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:44.915585 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:44.915585 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:44.921235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:22:44.921235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:22:44.921235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:22:44.921235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:22:44.921235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 637/1265] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:22:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:22:44.921597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd40f3e-a95e-48c5-b6b6-ccae2b90d843 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:45.001738 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:45.020981 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd40f3e-a95e-48c5-b6b6-ccae2b90d843 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:45.054556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626545e-a6aa-4bb7-8d19-6340cbeae44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:45.055212 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626545e-a6aa-4bb7-8d19-6340cbeae44e None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:45.055498 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626545e-a6aa-4bb7-8d19-6340cbeae44e None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:45.055780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6626545e-a6aa-4bb7-8d19-6340cbeae44e None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:45.056524 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626545e-a6aa-4bb7-8d19-6340cbeae44e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:45.056833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626545e-a6aa-4bb7-8d19-6340cbeae44e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:45.058202 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6626545e-a6aa-4bb7-8d19-6340cbeae44e None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:45.071901 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 629/1266] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:22:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:45.077528 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6eef62-ec65-43b5-a8a2-22c9cf955a59 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:45.174480 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6eef62-ec65-43b5-a8a2-22c9cf955a59 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:45.194031 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:45.207800 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d5fdd-5eb7-42d4-a796-d2f215cd181d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:45.208616 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d5fdd-5eb7-42d4-a796-d2f215cd181d None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:45.208903 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d5fdd-5eb7-42d4-a796-d2f215cd181d None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:45.209181 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d5fdd-5eb7-42d4-a796-d2f215cd181d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:45.216426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d5fdd-5eb7-42d4-a796-d2f215cd181d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:45.217857 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d5fdd-5eb7-42d4-a796-d2f215cd181d None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:45.232221 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-a58d5fdd-5eb7-42d4-a796-d2f215cd181d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 12:22:45.232221 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:45.232221 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:45.232221 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:45.232221 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:45.232221 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:45.232221 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:45.232221 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:45.232221 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:45.232221 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:45.232221 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:45.232221 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:45.232221 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:22:45.232221 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:22:45.232221 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:22:45.232221 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:22:45.233344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:45.233344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:45.233344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:22:45.233344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:22:45.233344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:22:45.233344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:22:45.233344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:45.233344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:45.233344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:45.233344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:45.233344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:45.233344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:22:45.233344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:45.233344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:45.233344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:45.233344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:45.233344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:45.233344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:45.239561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:45.239561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:45.239561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:22:45.239561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:22:45.239561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:22:45.239561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:22:45.239561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:22:45.239561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:22:45.239561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 12:22:45.239561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:22:45.239561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 638/1267] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:22:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:22:45.240160 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626545e-a6aa-4bb7-8d19-6340cbeae44e None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:45.335678 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6626545e-a6aa-4bb7-8d19-6340cbeae44e None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:45.368877 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44147e5d-de97-4725-bf42-24ec5b3a055f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:45.369530 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44147e5d-de97-4725-bf42-24ec5b3a055f None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:45.369814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44147e5d-de97-4725-bf42-24ec5b3a055f None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:45.370091 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44147e5d-de97-4725-bf42-24ec5b3a055f None admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:45.370915 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44147e5d-de97-4725-bf42-24ec5b3a055f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:45.371228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44147e5d-de97-4725-bf42-24ec5b3a055f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:45.372689 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44147e5d-de97-4725-bf42-24ec5b3a055f None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:45.381102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:45.389889 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 630/1268] 10.222.0.16 () {56 vars in 1143 bytes} [Mon May 23 12:22:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:45.395687 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d5fdd-5eb7-42d4-a796-d2f215cd181d None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:45.493293 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d5fdd-5eb7-42d4-a796-d2f215cd181d None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:45.524933 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c9b3ab-245d-4875-a133-e2582a712ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:45.525700 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9b3ab-245d-4875-a133-e2582a712ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:45.525982 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9b3ab-245d-4875-a133-e2582a712ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:45.526259 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c9b3ab-245d-4875-a133-e2582a712ea6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:45.536968 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c9b3ab-245d-4875-a133-e2582a712ea6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:45.538389 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c9b3ab-245d-4875-a133-e2582a712ea6 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:45.555543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-f1c9b3ab-245d-4875-a133-e2582a712ea6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:22:45.555543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:45.555543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:45.555543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:45.555543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:45.555543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:45.555543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:45.555543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:45.555543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:45.555543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:45.555543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:45.555543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:45.555543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:22:45.555543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:22:45.555543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:22:45.556648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:22:45.556648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:45.556648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:45.556648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:45.556648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:45.556648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:45.556648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:45.556648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:45.556648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:22:45.556648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:45.556648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:45.556648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:45.556648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:45.556648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:45.556648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:45.556648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:45.556648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:45.557745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:22:45.557745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:22:45.557745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:22:45.557745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:22:45.557745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 639/1269] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:22:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:22:45.562245 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44147e5d-de97-4725-bf42-24ec5b3a055f None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:45.658805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44147e5d-de97-4725-bf42-24ec5b3a055f None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:45.691063 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b124b632-d214-4ea0-ad77-2b20c5bd92e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:45.691705 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124b632-d214-4ea0-ad77-2b20c5bd92e0 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:45.691988 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124b632-d214-4ea0-ad77-2b20c5bd92e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:45.692288 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124b632-d214-4ea0-ad77-2b20c5bd92e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:45.693001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b124b632-d214-4ea0-ad77-2b20c5bd92e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:45.693312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b124b632-d214-4ea0-ad77-2b20c5bd92e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:45.694925 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b124b632-d214-4ea0-ad77-2b20c5bd92e0 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:45.707146 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 631/1270] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:22:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:45.712760 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c9b3ab-245d-4875-a133-e2582a712ea6 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:45.743814 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:45.813166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c9b3ab-245d-4875-a133-e2582a712ea6 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:45.847497 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2f5cc-6169-4889-8932-20182b792295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:45.848320 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2f5cc-6169-4889-8932-20182b792295 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:45.848605 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2f5cc-6169-4889-8932-20182b792295 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:45.848881 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2f5cc-6169-4889-8932-20182b792295 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:45.855337 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd2f5cc-6169-4889-8932-20182b792295 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:45.856811 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd2f5cc-6169-4889-8932-20182b792295 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:45.871764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-0cd2f5cc-6169-4889-8932-20182b792295 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:22:45.871764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:45.871764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:45.871764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:45.871764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:45.871764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:45.871764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:45.871764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:45.871764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:45.871764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:45.871764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:45.871764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:45.871764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:22:45.871764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:22:45.871764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:22:45.872857 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:22:45.872857 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:45.872857 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:45.872857 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:45.872857 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:45.872857 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:45.872857 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:45.872857 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:45.872857 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:22:45.872857 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:45.872857 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:45.872857 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:45.872857 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:45.872857 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:45.872857 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:45.872857 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:45.872857 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:45.874280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:22:45.874280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:22:45.874280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:22:45.874280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:22:45.874280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:22:45.874280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:22:45.874280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:22:45.874280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:22:45.874280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 640/1271] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:22:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:22:45.878821 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b124b632-d214-4ea0-ad77-2b20c5bd92e0 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:45.935645 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:45.977011 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b124b632-d214-4ea0-ad77-2b20c5bd92e0 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:46.009456 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22c099-23a9-4b78-93eb-8733061cc9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:46.010034 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22c099-23a9-4b78-93eb-8733061cc9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:46.010317 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22c099-23a9-4b78-93eb-8733061cc9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:46.010593 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22c099-23a9-4b78-93eb-8733061cc9d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:46.011363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22c099-23a9-4b78-93eb-8733061cc9d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:46.011672 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22c099-23a9-4b78-93eb-8733061cc9d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:46.013126 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22c099-23a9-4b78-93eb-8733061cc9d0 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:46.026432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 632/1272] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:22:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:46.032313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2f5cc-6169-4889-8932-20182b792295 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:46.120569 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:46.129648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2f5cc-6169-4889-8932-20182b792295 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:46.162263 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4d3928-f115-45ad-b3c5-784c22e7cca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:46.163138 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d3928-f115-45ad-b3c5-784c22e7cca4 None admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:46.163419 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d3928-f115-45ad-b3c5-784c22e7cca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:46.163694 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d3928-f115-45ad-b3c5-784c22e7cca4 None admin] PATH_INFO: `/v3/projects/839585501f414e1ab02501422510e54f/users/ebc0f137e0d24dd2a460689bc17a1b98/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:46.165629 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4d3928-f115-45ad-b3c5-784c22e7cca4 None admin] RBAC: Authorizing `identity:create_grant(project_id=839585501f414e1ab02501422510e54f, user_id=ebc0f137e0d24dd2a460689bc17a1b98, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:46.167051 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4d3928-f115-45ad-b3c5-784c22e7cca4 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:46.179141 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 641/1273] 10.222.0.16 () {58 vars in 1439 bytes} [Mon May 23 12:22:46 2022] PUT /identity/v3/projects/839585501f414e1ab02501422510e54f/users/ebc0f137e0d24dd2a460689bc17a1b98/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:22:48.343495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1cbef-dd32-4e9c-89f3-79b2861155e7 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:48.344026 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1cbef-dd32-4e9c-89f3-79b2861155e7 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:48.344026 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c1cbef-dd32-4e9c-89f3-79b2861155e7 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:48.345927 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 633/1274] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:22:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:22:48.351348 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bd85c-5561-4a83-9bf9-e032dc783036 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:48.351763 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bd85c-5561-4a83-9bf9-e032dc783036 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:48.351904 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bd85c-5561-4a83-9bf9-e032dc783036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:48.368964 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-c18bd85c-5561-4a83-9bf9-e032dc783036 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:22:48.370134 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c18bd85c-5561-4a83-9bf9-e032dc783036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:22:48.434589 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 642/1275] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:22:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:22:48.457971 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c1cbef-dd32-4e9c-89f3-79b2861155e7 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:48.556951 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c1cbef-dd32-4e9c-89f3-79b2861155e7 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:48.573752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:48.590735 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb6065-ecee-416c-81de-780a4262b4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:48.591514 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb6065-ecee-416c-81de-780a4262b4f6 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:48.591798 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb6065-ecee-416c-81de-780a4262b4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:48.592075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb6065-ecee-416c-81de-780a4262b4f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:48.603028 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb6065-ecee-416c-81de-780a4262b4f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 12:22:48.610507 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-9beb6065-ecee-416c-81de-780a4262b4f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:22:48.610507 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:48.610507 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:48.610507 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:48.610507 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:48.610507 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:48.610507 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:48.610507 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:48.610507 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:48.610507 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:48.610507 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:48.610507 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:48.610507 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 12:22:48.610507 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 12:22:48.610507 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 12:22:48.610507 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 12:22:48.611754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 12:22:48.611754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 12:22:48.611754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 12:22:48.611754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 12:22:48.611754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:48.611754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:48.611754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:48.611754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:48.611754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:48.611754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:48.611754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:48.611754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:22:48.611754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:48.611754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:48.611754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:48.611754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:48.612719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:48.612719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:48.612719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:48.612719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:48.612719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 12:22:48.612719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 12:22:48.612719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 12:22:48.612719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 12:22:48.612719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 12:22:48.612719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 12:22:48.612719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 12:22:48.612719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:22:48.612719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 634/1276] 10.222.0.16 () {56 vars in 1096 bytes} [Mon May 23 12:22:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:22:48.617707 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18bd85c-5561-4a83-9bf9-e032dc783036 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:48.695810 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:48.714441 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18bd85c-5561-4a83-9bf9-e032dc783036 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:48.747049 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93959540-64a4-42e6-87ce-4abc9bea2b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:48.747702 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93959540-64a4-42e6-87ce-4abc9bea2b38 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:48.747993 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93959540-64a4-42e6-87ce-4abc9bea2b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:48.748302 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93959540-64a4-42e6-87ce-4abc9bea2b38 None admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:48.749041 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93959540-64a4-42e6-87ce-4abc9bea2b38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:48.749350 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93959540-64a4-42e6-87ce-4abc9bea2b38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:48.750703 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93959540-64a4-42e6-87ce-4abc9bea2b38 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:48.759118 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 643/1277] 10.222.0.16 () {56 vars in 1099 bytes} [Mon May 23 12:22:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:48.764885 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb6065-ecee-416c-81de-780a4262b4f6 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:48.884442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb6065-ecee-416c-81de-780a4262b4f6 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:48.917459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3fec06-4aaa-4719-8b10-a5c116bd1961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:48.918238 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3fec06-4aaa-4719-8b10-a5c116bd1961 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:48.918553 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3fec06-4aaa-4719-8b10-a5c116bd1961 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:48.918833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3fec06-4aaa-4719-8b10-a5c116bd1961 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:48.929335 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3fec06-4aaa-4719-8b10-a5c116bd1961 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:48.930784 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3fec06-4aaa-4719-8b10-a5c116bd1961 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:48.948299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-8d3fec06-4aaa-4719-8b10-a5c116bd1961 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:22:48.948299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:48.948299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:48.948299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:48.948299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:48.948299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:48.948299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:48.948299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:48.948299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:48.948299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:48.948299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:48.948299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:48.948299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:22:48.948299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:22:48.948299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:22:48.949329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:22:48.949329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:48.949329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:48.949329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:48.949329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:48.949329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:48.949329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:48.949329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:48.949329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:22:48.949329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:48.949329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:48.949329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:48.949329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:48.949329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:48.949329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:48.949329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:48.949329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:48.950586 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:22:48.950586 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:22:48.950586 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:22:48.950586 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:22:48.950586 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 635/1278] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:22:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:22:48.955968 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93959540-64a4-42e6-87ce-4abc9bea2b38 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:49.052227 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93959540-64a4-42e6-87ce-4abc9bea2b38 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:49.088317 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80455e90-cb8c-4a48-a3c2-efc0a78720cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:49.088970 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80455e90-cb8c-4a48-a3c2-efc0a78720cf None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:49.089254 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80455e90-cb8c-4a48-a3c2-efc0a78720cf None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:49.089529 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80455e90-cb8c-4a48-a3c2-efc0a78720cf None admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:49.090243 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80455e90-cb8c-4a48-a3c2-efc0a78720cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:49.090553 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80455e90-cb8c-4a48-a3c2-efc0a78720cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:49.091990 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80455e90-cb8c-4a48-a3c2-efc0a78720cf None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:49.104165 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 644/1279] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:22:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:49.109769 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3fec06-4aaa-4719-8b10-a5c116bd1961 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:49.143001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:49.210639 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3fec06-4aaa-4719-8b10-a5c116bd1961 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:49.242521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac086792-ef21-4d33-b72f-e07cffaa5b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:49.243311 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac086792-ef21-4d33-b72f-e07cffaa5b6d None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:49.243597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac086792-ef21-4d33-b72f-e07cffaa5b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:49.243874 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac086792-ef21-4d33-b72f-e07cffaa5b6d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:49.251140 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac086792-ef21-4d33-b72f-e07cffaa5b6d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:49.253174 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac086792-ef21-4d33-b72f-e07cffaa5b6d None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:49.267254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-ac086792-ef21-4d33-b72f-e07cffaa5b6d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 12:22:49.267254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:49.267254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:49.267254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:49.267254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:49.267254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:49.267254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:49.267254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:49.267254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:49.267254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:49.267254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:49.267254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:49.267254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 12:22:49.267254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 12:22:49.267254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 12:22:49.267254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 12:22:49.268601 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:49.268601 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:49.268601 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 12:22:49.268601 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:22:49.268601 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 12:22:49.268601 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 12:22:49.268601 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:49.268601 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:49.268601 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:49.268601 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:49.268601 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:49.268601 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:22:49.268601 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:49.268601 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:49.268601 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:49.268601 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:49.268601 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:49.268601 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:49.269779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:49.269779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:49.269779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 12:22:49.269779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 12:22:49.269779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 12:22:49.269779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 12:22:49.269779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 12:22:49.269779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 12:22:49.269779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 12:22:49.269779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:22:49.269779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 636/1280] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:22:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:22:49.274318 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80455e90-cb8c-4a48-a3c2-efc0a78720cf None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:49.278027 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:49.378009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80455e90-cb8c-4a48-a3c2-efc0a78720cf None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:49.411400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa74c3-2f44-4e3b-8a2c-040978692d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:49.411993 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa74c3-2f44-4e3b-8a2c-040978692d68 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:49.412306 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa74c3-2f44-4e3b-8a2c-040978692d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:49.412588 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa74c3-2f44-4e3b-8a2c-040978692d68 None admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:49.413408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fa74c3-2f44-4e3b-8a2c-040978692d68 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:49.413719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fa74c3-2f44-4e3b-8a2c-040978692d68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:49.415125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fa74c3-2f44-4e3b-8a2c-040978692d68 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:49.431729 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 645/1281] 10.222.0.16 () {56 vars in 1143 bytes} [Mon May 23 12:22:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:49.437311 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac086792-ef21-4d33-b72f-e07cffaa5b6d None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:49.526289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:49.536767 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac086792-ef21-4d33-b72f-e07cffaa5b6d None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:49.568914 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a36d55-5e49-4d72-8812-f9ec15421fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:49.569549 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a36d55-5e49-4d72-8812-f9ec15421fce None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:49.569833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a36d55-5e49-4d72-8812-f9ec15421fce None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:49.570110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a36d55-5e49-4d72-8812-f9ec15421fce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:49.634881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a36d55-5e49-4d72-8812-f9ec15421fce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:49.636362 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a36d55-5e49-4d72-8812-f9ec15421fce None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:49.729450 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:49.739854 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 637/1282] 10.222.0.16 () {58 vars in 1282 bytes} [Mon May 23 12:22:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 12:22:49.745904 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa74c3-2f44-4e3b-8a2c-040978692d68 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:49.811377 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:49.849911 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa74c3-2f44-4e3b-8a2c-040978692d68 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:49.882871 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebcf8e2-8da6-48eb-87c1-8c660c7292fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:49.883655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcf8e2-8da6-48eb-87c1-8c660c7292fe None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:49.883944 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcf8e2-8da6-48eb-87c1-8c660c7292fe None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:49.884259 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcf8e2-8da6-48eb-87c1-8c660c7292fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:49.894674 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebcf8e2-8da6-48eb-87c1-8c660c7292fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:49.896107 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebcf8e2-8da6-48eb-87c1-8c660c7292fe None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:49.914045 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-6ebcf8e2-8da6-48eb-87c1-8c660c7292fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:22:49.914045 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:49.914045 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:49.914045 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:49.914045 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:49.914045 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:49.914045 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:49.914045 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:49.914045 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:49.914045 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:49.914045 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:49.914045 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:49.914045 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 12:22:49.914045 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 12:22:49.914045 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 12:22:49.915102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 12:22:49.915102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:49.915102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:49.915102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:49.915102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:49.915102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:49.915102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:49.915102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:49.915102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:22:49.915102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:49.915102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:49.915102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:49.915102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:49.915102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:49.915102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:49.915102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:49.915102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:49.916254 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 12:22:49.916254 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 12:22:49.916254 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 12:22:49.916254 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:22:49.916254 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 646/1283] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:22:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:22:49.921135 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a36d55-5e49-4d72-8812-f9ec15421fce None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:50.018741 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a36d55-5e49-4d72-8812-f9ec15421fce None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:50.052813 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad92bf4-7d2c-4db7-9c8a-b463fee0ae59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:50.053464 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad92bf4-7d2c-4db7-9c8a-b463fee0ae59 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:50.053752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad92bf4-7d2c-4db7-9c8a-b463fee0ae59 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:50.054034 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad92bf4-7d2c-4db7-9c8a-b463fee0ae59 None admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:50.054749 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad92bf4-7d2c-4db7-9c8a-b463fee0ae59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:50.055058 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad92bf4-7d2c-4db7-9c8a-b463fee0ae59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:50.056460 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad92bf4-7d2c-4db7-9c8a-b463fee0ae59 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:50.068290 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 638/1284] 10.222.0.16 () {56 vars in 1109 bytes} [Mon May 23 12:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:50.074066 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebcf8e2-8da6-48eb-87c1-8c660c7292fe None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:50.170119 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebcf8e2-8da6-48eb-87c1-8c660c7292fe None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:50.184265 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:50.204501 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101231ba-bdf6-4b74-b94c-1dd0a2365367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:50.205383 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101231ba-bdf6-4b74-b94c-1dd0a2365367 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:50.205667 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101231ba-bdf6-4b74-b94c-1dd0a2365367 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:50.205944 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101231ba-bdf6-4b74-b94c-1dd0a2365367 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:50.212386 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101231ba-bdf6-4b74-b94c-1dd0a2365367 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:50.213817 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101231ba-bdf6-4b74-b94c-1dd0a2365367 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:50.227266 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application [None req-101231ba-bdf6-4b74-b94c-1dd0a2365367 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:22:50.227266 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:50.227266 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:50.227266 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:50.227266 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:50.227266 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:50.227266 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:50.227266 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:50.227266 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:50.227266 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:50.227266 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:50.227266 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:50.227266 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 12:22:50.227266 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 12:22:50.227266 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 12:22:50.228336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 12:22:50.228336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:50.228336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:50.228336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:50.228336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:50.228336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:50.228336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:50.228336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:50.228336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application with Lock( May 23 12:22:50.228336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:50.228336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:50.228336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:50.228336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:50.228336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:50.228336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:50.228336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:50.228336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:50.229620 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 12:22:50.229620 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 12:22:50.229620 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 12:22:50.229620 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 12:22:50.229620 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 12:22:50.229620 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 12:22:50.229620 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 12:22:50.229620 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: ERROR keystone.server.flask.application May 23 12:22:50.229620 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 647/1285] 10.222.0.16 () {56 vars in 1111 bytes} [Mon May 23 12:22:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:22:50.234121 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad92bf4-7d2c-4db7-9c8a-b463fee0ae59 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:50.283234 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:50.332499 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad92bf4-7d2c-4db7-9c8a-b463fee0ae59 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:50.365209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5336fd27-1cee-4ba3-882c-bef08ea03f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:50.365854 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336fd27-1cee-4ba3-882c-bef08ea03f71 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:50.366142 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336fd27-1cee-4ba3-882c-bef08ea03f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:50.366421 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336fd27-1cee-4ba3-882c-bef08ea03f71 None admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:50.367195 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5336fd27-1cee-4ba3-882c-bef08ea03f71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:50.367507 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5336fd27-1cee-4ba3-882c-bef08ea03f71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:50.368964 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5336fd27-1cee-4ba3-882c-bef08ea03f71 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:50.380556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 639/1286] 10.222.0.16 () {56 vars in 1148 bytes} [Mon May 23 12:22:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:50.386736 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101231ba-bdf6-4b74-b94c-1dd0a2365367 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:50.483067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101231ba-bdf6-4b74-b94c-1dd0a2365367 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:50.515975 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae06a81-7b37-4557-8e18-242020902a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:50.516646 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae06a81-7b37-4557-8e18-242020902a05 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:50.516934 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae06a81-7b37-4557-8e18-242020902a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:50.517788 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae06a81-7b37-4557-8e18-242020902a05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:50.518609 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae06a81-7b37-4557-8e18-242020902a05 None admin] RBAC: Adding query filter params (role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8, scope.project.id=839585501f414e1ab02501422510e54f, user.id=ebc0f137e0d24dd2a460689bc17a1b98) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:50.518927 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae06a81-7b37-4557-8e18-242020902a05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:50.520364 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae06a81-7b37-4557-8e18-242020902a05 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:50.526003 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 648/1287] 10.222.0.16 () {56 vars in 1374 bytes} [Mon May 23 12:22:50 2022] GET /identity/v3/role_assignments?user.id=ebc0f137e0d24dd2a460689bc17a1b98&scope.project.id=839585501f414e1ab02501422510e54f&role.id=4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:52.736472 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6988cc7-7fd4-482f-bc99-97cfd2f5fb63 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:52.737002 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6988cc7-7fd4-482f-bc99-97cfd2f5fb63 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:52.737408 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6988cc7-7fd4-482f-bc99-97cfd2f5fb63 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:52.739735 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 640/1288] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:22:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:22:52.745144 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfa624-cbbb-4d7a-a0ef-3cbf84c6a78b None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:52.745786 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfa624-cbbb-4d7a-a0ef-3cbf84c6a78b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:52.745786 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfa624-cbbb-4d7a-a0ef-3cbf84c6a78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:52.763546 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-87cfa624-cbbb-4d7a-a0ef-3cbf84c6a78b None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:22:52.764770 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-87cfa624-cbbb-4d7a-a0ef-3cbf84c6a78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:22:52.796008 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:52.846969 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 649/1289] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:22:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 12:22:52.887543 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6988cc7-7fd4-482f-bc99-97cfd2f5fb63 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:52.994070 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6988cc7-7fd4-482f-bc99-97cfd2f5fb63 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:53.005038 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:53.028825 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20897a-db11-4297-a08b-0b781f6152d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:53.029610 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20897a-db11-4297-a08b-0b781f6152d6 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:53.029891 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20897a-db11-4297-a08b-0b781f6152d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:53.030167 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20897a-db11-4297-a08b-0b781f6152d6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:53.030812 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f20897a-db11-4297-a08b-0b781f6152d6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:53.032187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f20897a-db11-4297-a08b-0b781f6152d6 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:53.044442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-0f20897a-db11-4297-a08b-0b781f6152d6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 23 12:22:53.044442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:53.044442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 12:22:53.044442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 12:22:53.044442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 12:22:53.044442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 12:22:53.044442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 12:22:53.044442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:22:53.044442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 12:22:53.044442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:22:53.044442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:22:53.044442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:22:53.044442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:53.044442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:53.044442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:53.044442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:53.044442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:53.045487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:53.045487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:53.045487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:53.045487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:53.045487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:53.045487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:53.045487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 12:22:53.045487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 12:22:53.045487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 12:22:53.045487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 12:22:53.045487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:53.045487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:53.045487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:53.045487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:53.045487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:53.045487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:53.046431 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:53.046431 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:22:53.046431 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:53.046431 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:53.046431 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:53.046431 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:53.046431 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:53.046431 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:53.046431 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:53.046431 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:53.046431 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 12:22:53.046431 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:22:53.046431 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 12:22:53.046431 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:22:53.046431 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 641/1290] 10.222.0.16 () {56 vars in 1117 bytes} [Mon May 23 12:22:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:22:53.052761 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfa624-cbbb-4d7a-a0ef-3cbf84c6a78b None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:53.155381 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfa624-cbbb-4d7a-a0ef-3cbf84c6a78b None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:53.191471 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3932e-9849-4a5f-908c-bbf0e62fe6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:53.192130 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3932e-9849-4a5f-908c-bbf0e62fe6cf None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:53.192444 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3932e-9849-4a5f-908c-bbf0e62fe6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:53.192723 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3932e-9849-4a5f-908c-bbf0e62fe6cf None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:53.193420 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3932e-9849-4a5f-908c-bbf0e62fe6cf None admin] RBAC: Adding query filter params (name=placement) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:53.193978 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3932e-9849-4a5f-908c-bbf0e62fe6cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:53.195342 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3932e-9849-4a5f-908c-bbf0e62fe6cf None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:53.200657 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 650/1291] 10.222.0.16 () {56 vars in 1116 bytes} [Mon May 23 12:22:53 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:53.207091 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20897a-db11-4297-a08b-0b781f6152d6 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:53.304530 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20897a-db11-4297-a08b-0b781f6152d6 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:53.337670 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116595a-29d9-41d8-a91e-919688b80e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:53.338322 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116595a-29d9-41d8-a91e-919688b80e79 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:53.338638 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116595a-29d9-41d8-a91e-919688b80e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:53.338921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116595a-29d9-41d8-a91e-919688b80e79 None admin] PATH_INFO: `/v3/services` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:53.339624 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116595a-29d9-41d8-a91e-919688b80e79 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:53.339941 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116595a-29d9-41d8-a91e-919688b80e79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:53.341346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116595a-29d9-41d8-a91e-919688b80e79 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:53.346430 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 642/1292] 10.222.0.16 () {56 vars in 1116 bytes} [Mon May 23 12:22:53 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:55.547257 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c89c9-e0e7-4d89-8b84-28387a36f716 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:55.547814 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c89c9-e0e7-4d89-8b84-28387a36f716 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:55.547814 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c89c9-e0e7-4d89-8b84-28387a36f716 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:55.549624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 651/1293] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:22:55.554681 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bae1f-4af2-4c91-8889-c3de47b57d60 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:55.555056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bae1f-4af2-4c91-8889-c3de47b57d60 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:55.555246 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bae1f-4af2-4c91-8889-c3de47b57d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:55.571948 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-e71bae1f-4af2-4c91-8889-c3de47b57d60 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:22:55.573162 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-e71bae1f-4af2-4c91-8889-c3de47b57d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:22:55.602484 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:55.639639 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 643/1294] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:22:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:22:55.662595 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c89c9-e0e7-4d89-8b84-28387a36f716 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:55.693103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:55.761042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c89c9-e0e7-4d89-8b84-28387a36f716 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:55.796301 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81160f54-f089-44a1-9656-9e685f5027c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:55.797164 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81160f54-f089-44a1-9656-9e685f5027c5 None admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:55.797442 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81160f54-f089-44a1-9656-9e685f5027c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:55.797694 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81160f54-f089-44a1-9656-9e685f5027c5 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:55.798570 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81160f54-f089-44a1-9656-9e685f5027c5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:55.800786 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81160f54-f089-44a1-9656-9e685f5027c5 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:55.818513 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 652/1295] 10.222.0.16 () {60 vars in 1141 bytes} [Mon May 23 12:22:55 2022] POST /identity/v3/services => generated 246 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:22:57.998625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef16a0-180b-45d6-8549-e8966261ebe5 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:57.999202 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef16a0-180b-45d6-8549-e8966261ebe5 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:57.999202 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef16a0-180b-45d6-8549-e8966261ebe5 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:58.000564 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 644/1296] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:22:58.006323 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabffd9-779d-4414-8d90-134407a906c3 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:58.006742 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabffd9-779d-4414-8d90-134407a906c3 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:58.006882 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabffd9-779d-4414-8d90-134407a906c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:58.024319 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-eaabffd9-779d-4414-8d90-134407a906c3 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:22:58.025510 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-eaabffd9-779d-4414-8d90-134407a906c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:22:58.096551 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 653/1297] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:22:58 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:22:58.119063 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef16a0-180b-45d6-8549-e8966261ebe5 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:58.217808 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef16a0-180b-45d6-8549-e8966261ebe5 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:58.250247 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f4343-c71e-4f23-907d-d595b56d8a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:58.250972 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f4343-c71e-4f23-907d-d595b56d8a19 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:58.251758 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f4343-c71e-4f23-907d-d595b56d8a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:58.252037 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f4343-c71e-4f23-907d-d595b56d8a19 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:58.252728 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f4343-c71e-4f23-907d-d595b56d8a19 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:58.254115 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f4343-c71e-4f23-907d-d595b56d8a19 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:58.265604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-3b7f4343-c71e-4f23-907d-d595b56d8a19 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 23 12:22:58.265604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:58.265604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 12:22:58.265604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 12:22:58.265604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 12:22:58.265604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 12:22:58.265604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 12:22:58.265604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:22:58.265604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 12:22:58.265604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:22:58.265604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:22:58.265604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:22:58.265604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:22:58.265604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:22:58.265604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:22:58.265604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:22:58.265604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:22:58.266795 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:22:58.266795 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:22:58.266795 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:22:58.266795 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:22:58.266795 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:22:58.266795 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:22:58.266795 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 12:22:58.266795 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 12:22:58.266795 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 12:22:58.266795 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 12:22:58.266795 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:22:58.266795 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:22:58.266795 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:22:58.266795 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:22:58.266795 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:22:58.266795 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:22:58.268774 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:22:58.268774 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:22:58.268774 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:22:58.268774 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:22:58.268774 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:22:58.268774 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:22:58.268774 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:22:58.268774 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:22:58.268774 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:22:58.268774 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:22:58.268774 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 12:22:58.268774 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:22:58.268774 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 12:22:58.268774 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:22:58.268774 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 645/1298] 10.222.0.16 () {56 vars in 1117 bytes} [Mon May 23 12:22:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:22:58.273356 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabffd9-779d-4414-8d90-134407a906c3 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:58.291138 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:58.370258 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabffd9-779d-4414-8d90-134407a906c3 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:58.402994 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238707b9-cd56-458e-a163-8b84c9433f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:58.403650 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238707b9-cd56-458e-a163-8b84c9433f98 None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:58.403936 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238707b9-cd56-458e-a163-8b84c9433f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:58.404239 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238707b9-cd56-458e-a163-8b84c9433f98 None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:58.404945 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238707b9-cd56-458e-a163-8b84c9433f98 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:58.405257 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238707b9-cd56-458e-a163-8b84c9433f98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:58.406617 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238707b9-cd56-458e-a163-8b84c9433f98 None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:58.411625 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 654/1299] 10.222.0.16 () {56 vars in 1116 bytes} [Mon May 23 12:22:58 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:58.417458 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f4343-c71e-4f23-907d-d595b56d8a19 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:58.449307 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:58.516306 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f4343-c71e-4f23-907d-d595b56d8a19 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:58.548698 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06700ca1-477b-4d3d-af57-b5d4dbff6bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:58.549422 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06700ca1-477b-4d3d-af57-b5d4dbff6bad None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:58.549710 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06700ca1-477b-4d3d-af57-b5d4dbff6bad None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:58.549994 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06700ca1-477b-4d3d-af57-b5d4dbff6bad None admin] PATH_INFO: `/v3/endpoints` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:58.550781 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06700ca1-477b-4d3d-af57-b5d4dbff6bad None admin] RBAC: Adding query filter params (interface=public, service_id=1c3ab91519ad4ace8222948730ab0f21, region_id=RegionOne) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:22:58.551099 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06700ca1-477b-4d3d-af57-b5d4dbff6bad None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:58.552493 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06700ca1-477b-4d3d-af57-b5d4dbff6bad None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:58.557389 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 646/1300] 10.222.0.16 () {56 vars in 1251 bytes} [Mon May 23 12:22:58 2022] GET /identity/v3/endpoints?service_id=1c3ab91519ad4ace8222948730ab0f21&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:22:58.562888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238707b9-cd56-458e-a163-8b84c9433f98 None admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:22:58.643147 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:22:58.661628 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238707b9-cd56-458e-a163-8b84c9433f98 None admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:22:58.694653 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35702911-fb19-47c1-9c2a-e0a66ae3128e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:22:58.695307 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35702911-fb19-47c1-9c2a-e0a66ae3128e None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:22:58.695591 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35702911-fb19-47c1-9c2a-e0a66ae3128e None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:22:58.695867 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35702911-fb19-47c1-9c2a-e0a66ae3128e None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:22:58.696579 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35702911-fb19-47c1-9c2a-e0a66ae3128e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:22:58.697934 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35702911-fb19-47c1-9c2a-e0a66ae3128e None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:22:58.704630 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 655/1301] 10.222.0.16 () {56 vars in 1087 bytes} [Mon May 23 12:22:58 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:23:00.908040 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3ab47-e111-4855-8f63-90bac8501997 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:23:00.908620 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3ab47-e111-4855-8f63-90bac8501997 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:23:00.908714 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3ab47-e111-4855-8f63-90bac8501997 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:23:00.910397 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 647/1302] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:23:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:23:00.916228 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa53fe-1036-4fec-9f60-1a976d45d3b6 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:23:00.916636 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa53fe-1036-4fec-9f60-1a976d45d3b6 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:23:00.916783 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa53fe-1036-4fec-9f60-1a976d45d3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:23:00.933397 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-d5aa53fe-1036-4fec-9f60-1a976d45d3b6 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:23:00.934561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d5aa53fe-1036-4fec-9f60-1a976d45d3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:23:00.999887 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 656/1303] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:23:00 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 12:23:01.024013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3ab47-e111-4855-8f63-90bac8501997 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:23:01.116773 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:23:01.123927 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3ab47-e111-4855-8f63-90bac8501997 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:23:01.161317 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206273d2-a912-4e68-ba5a-c184bc079f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:23:01.162167 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206273d2-a912-4e68-ba5a-c184bc079f60 None admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:23:01.162521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206273d2-a912-4e68-ba5a-c184bc079f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:23:01.162816 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206273d2-a912-4e68-ba5a-c184bc079f60 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:23:01.163722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206273d2-a912-4e68-ba5a-c184bc079f60 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:23:01.165628 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206273d2-a912-4e68-ba5a-c184bc079f60 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:23:01.177469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application [None req-206273d2-a912-4e68-ba5a-c184bc079f60 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 23 12:23:01.177469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:23:01.177469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 12:23:01.177469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 12:23:01.177469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 12:23:01.177469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 12:23:01.177469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 12:23:01.177469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:23:01.177469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 12:23:01.177469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:23:01.177469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 12:23:01.177469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:23:01.177469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 12:23:01.177469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 12:23:01.177469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 12:23:01.177469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 12:23:01.177469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 12:23:01.178647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 12:23:01.178647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 12:23:01.178647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 12:23:01.178647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 12:23:01.178647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 12:23:01.178647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 12:23:01.178647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 12:23:01.178647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 12:23:01.178647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 12:23:01.178647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 12:23:01.178647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 12:23:01.178647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 12:23:01.178647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 12:23:01.178647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 12:23:01.178647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 12:23:01.178647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.get_or_create( May 23 12:23:01.179677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 12:23:01.179677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application with Lock( May 23 12:23:01.179677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 12:23:01.179677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self._enter() May 23 12:23:01.179677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 12:23:01.179677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 12:23:01.179677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 12:23:01.179677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application return self.creator() May 23 12:23:01.179677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 12:23:01.179677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application created_value = creator( May 23 12:23:01.179677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 12:23:01.179677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 12:23:01.179677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 12:23:01.179677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: ERROR keystone.server.flask.application May 23 12:23:01.179677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 648/1304] 10.222.0.16 () {56 vars in 1117 bytes} [Mon May 23 12:23:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 12:23:01.184986 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa53fe-1036-4fec-9f60-1a976d45d3b6 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:23:01.258755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:23:01.281552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa53fe-1036-4fec-9f60-1a976d45d3b6 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:23:01.315271 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b1dfc-6328-4563-97cb-47ace19282dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:23:01.315953 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b1dfc-6328-4563-97cb-47ace19282dc None admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:23:01.316262 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b1dfc-6328-4563-97cb-47ace19282dc None admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:23:01.316538 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b1dfc-6328-4563-97cb-47ace19282dc None admin] PATH_INFO: `/v3/services` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:23:01.317234 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85b1dfc-6328-4563-97cb-47ace19282dc None admin] RBAC: Adding query filter params (name=placement) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:23:01.317537 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85b1dfc-6328-4563-97cb-47ace19282dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:23:01.319148 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85b1dfc-6328-4563-97cb-47ace19282dc None admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:23:01.324255 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 657/1305] 10.222.0.16 () {56 vars in 1116 bytes} [Mon May 23 12:23:01 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:23:01.330108 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206273d2-a912-4e68-ba5a-c184bc079f60 None admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:23:01.427707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206273d2-a912-4e68-ba5a-c184bc079f60 None admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:23:01.460531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561722ba-d65e-43c4-99f2-ec48ebfd1c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:23:01.461173 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561722ba-d65e-43c4-99f2-ec48ebfd1c77 None admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:23:01.461461 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561722ba-d65e-43c4-99f2-ec48ebfd1c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:23:01.461742 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561722ba-d65e-43c4-99f2-ec48ebfd1c77 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:23:01.462484 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561722ba-d65e-43c4-99f2-ec48ebfd1c77 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:23:01.463865 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561722ba-d65e-43c4-99f2-ec48ebfd1c77 None admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:23:01.481456 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 649/1306] 10.222.0.16 () {60 vars in 1144 bytes} [Mon May 23 12:23:01 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:23:11.321282 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=30 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:23:11.477724 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:24:05.279197 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad5859-5ed8-45c8-8854-e23d41ca0241 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:05.279701 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad5859-5ed8-45c8-8854-e23d41ca0241 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:05.279701 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad5859-5ed8-45c8-8854-e23d41ca0241 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:05.285446 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 658/1307] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:24:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:24:05.287055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbf056-07b6-40eb-bc72-06726083f0a6 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:05.287890 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbf056-07b6-40eb-bc72-06726083f0a6 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:05.288377 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbf056-07b6-40eb-bc72-06726083f0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:05.368965 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-99bbf056-07b6-40eb-bc72-06726083f0a6 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:24:05.370341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-99bbf056-07b6-40eb-bc72-06726083f0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:05.479297 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 650/1308] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:24:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:24:05.528097 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757680e6-cf07-4835-a7a5-9151a080aa17 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:05.528358 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757680e6-cf07-4835-a7a5-9151a080aa17 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:05.528646 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757680e6-cf07-4835-a7a5-9151a080aa17 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:05.530419 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 659/1309] 10.222.0.16 () {54 vars in 964 bytes} [Mon May 23 12:24:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:24:05.537753 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d031c37-a6fd-46df-bd05-8e7ab6cb437f None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:05.537753 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d031c37-a6fd-46df-bd05-8e7ab6cb437f None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:05.538068 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d031c37-a6fd-46df-bd05-8e7ab6cb437f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:05.639860 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-8d031c37-a6fd-46df-bd05-8e7ab6cb437f None None] MFA Rules not processed for user `b6677be3ddf647e2b971bb3ba2c34a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:24:05.641857 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-8d031c37-a6fd-46df-bd05-8e7ab6cb437f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:05.668754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:24:05.772448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 651/1310] 10.222.0.16 () {58 vars in 1061 bytes} [Mon May 23 12:24:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:24:05.781828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757680e6-cf07-4835-a7a5-9151a080aa17 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:24:05.784201 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-757680e6-cf07-4835-a7a5-9151a080aa17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:05.942077 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757680e6-cf07-4835-a7a5-9151a080aa17 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:24:06.027104 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e6d181-4691-4844-a72c-1df6ab0b4472 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:24:06.028059 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6d181-4691-4844-a72c-1df6ab0b4472 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:06.029119 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6d181-4691-4844-a72c-1df6ab0b4472 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:06.029562 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6d181-4691-4844-a72c-1df6ab0b4472 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:06.031693 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e1e6d181-4691-4844-a72c-1df6ab0b4472 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:06.090991 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:24:06.150337 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e6d181-4691-4844-a72c-1df6ab0b4472 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:24:06.151857 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e6d181-4691-4844-a72c-1df6ab0b4472 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:24:06.173550 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 660/1311] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:24:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:24:08.768665 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c77415-71bc-4cdc-aa9b-be02771f5349 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:08.769237 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c77415-71bc-4cdc-aa9b-be02771f5349 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:08.769237 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c77415-71bc-4cdc-aa9b-be02771f5349 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:08.770590 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 652/1312] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:24:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:24:08.775870 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1cafdc-d6a9-4f0c-b167-8fcac65b3852 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:08.776343 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1cafdc-d6a9-4f0c-b167-8fcac65b3852 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:08.776619 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1cafdc-d6a9-4f0c-b167-8fcac65b3852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:08.866210 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-0b1cafdc-d6a9-4f0c-b167-8fcac65b3852 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:24:08.867439 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-0b1cafdc-d6a9-4f0c-b167-8fcac65b3852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:08.965866 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 661/1313] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:24:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:24:09.004683 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c77415-71bc-4cdc-aa9b-be02771f5349 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:24:09.006300 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-06c77415-71bc-4cdc-aa9b-be02771f5349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:09.118719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c77415-71bc-4cdc-aa9b-be02771f5349 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:24:09.192336 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:24:09.195400 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2b1e4-26bd-49c8-aa5b-5e6bdc7ad372 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:24:09.196249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2b1e4-26bd-49c8-aa5b-5e6bdc7ad372 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:09.196721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2b1e4-26bd-49c8-aa5b-5e6bdc7ad372 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:09.197296 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2b1e4-26bd-49c8-aa5b-5e6bdc7ad372 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:09.198942 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-9ec2b1e4-26bd-49c8-aa5b-5e6bdc7ad372 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:09.309075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec2b1e4-26bd-49c8-aa5b-5e6bdc7ad372 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:24:09.310447 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec2b1e4-26bd-49c8-aa5b-5e6bdc7ad372 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:24:09.330492 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 653/1314] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:24:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:24:11.929262 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea009c8a-0c4d-453e-9a0d-f9af1f9de637 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:11.929808 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea009c8a-0c4d-453e-9a0d-f9af1f9de637 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:11.929808 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea009c8a-0c4d-453e-9a0d-f9af1f9de637 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:11.931194 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 662/1315] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:24:11.936390 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95eff4f-6aa7-4452-a2e8-934a132abb62 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:11.936772 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95eff4f-6aa7-4452-a2e8-934a132abb62 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:11.936958 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95eff4f-6aa7-4452-a2e8-934a132abb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:12.004311 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-b95eff4f-6aa7-4452-a2e8-934a132abb62 None None] MFA Rules not processed for user `66b6b05f6db5476b801ab332f119cbdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:24:12.005538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-b95eff4f-6aa7-4452-a2e8-934a132abb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:12.064482 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:24:12.116459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 654/1316] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:24:11 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:24:12.153267 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea009c8a-0c4d-453e-9a0d-f9af1f9de637 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:24:12.154805 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ea009c8a-0c4d-453e-9a0d-f9af1f9de637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:12.165749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:24:12.263620 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea009c8a-0c4d-453e-9a0d-f9af1f9de637 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:24:12.344081 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f1250-9014-4048-94f8-4008a6516f28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:24:12.345298 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f1250-9014-4048-94f8-4008a6516f28 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:12.345692 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f1250-9014-4048-94f8-4008a6516f28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:12.346002 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f1250-9014-4048-94f8-4008a6516f28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:12.347648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a99f1250-9014-4048-94f8-4008a6516f28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:12.409682 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:24:12.458012 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99f1250-9014-4048-94f8-4008a6516f28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:24:12.459462 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99f1250-9014-4048-94f8-4008a6516f28 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:24:12.477932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 663/1317] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:24:12 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:24:15.274097 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15996d8-a695-437c-8afa-2d32dc4d2473 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:15.274633 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15996d8-a695-437c-8afa-2d32dc4d2473 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:15.274633 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15996d8-a695-437c-8afa-2d32dc4d2473 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:15.276005 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 655/1318] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:24:15.281110 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef831c-dabe-4dc5-b11f-e186bb375e6c None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:15.281502 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef831c-dabe-4dc5-b11f-e186bb375e6c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:15.281601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef831c-dabe-4dc5-b11f-e186bb375e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:15.346762 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-1cef831c-dabe-4dc5-b11f-e186bb375e6c None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:24:15.347964 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1cef831c-dabe-4dc5-b11f-e186bb375e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:15.448182 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 664/1319] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:24:15 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:24:15.501968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14718b3f-b32a-41f2-bca3-bfa47744f5fd None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:15.502253 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14718b3f-b32a-41f2-bca3-bfa47744f5fd None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:15.502532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14718b3f-b32a-41f2-bca3-bfa47744f5fd None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:15.503887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 656/1320] 10.222.0.16 () {54 vars in 964 bytes} [Mon May 23 12:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:24:15.509833 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d89874-7398-41e3-a997-d470f5143b4e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:15.510129 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d89874-7398-41e3-a997-d470f5143b4e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:15.510410 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d89874-7398-41e3-a997-d470f5143b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:15.533049 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:24:15.587945 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-16d89874-7398-41e3-a997-d470f5143b4e None None] MFA Rules not processed for user `b6677be3ddf647e2b971bb3ba2c34a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:24:15.589181 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-16d89874-7398-41e3-a997-d470f5143b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:15.662572 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 665/1321] 10.222.0.16 () {58 vars in 1061 bytes} [Mon May 23 12:24:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:24:15.671503 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14718b3f-b32a-41f2-bca3-bfa47744f5fd None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:24:15.673128 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-14718b3f-b32a-41f2-bca3-bfa47744f5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:15.776469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14718b3f-b32a-41f2-bca3-bfa47744f5fd None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:24:15.849590 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3848ec-417d-466f-ba32-a3228643eaf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:24:15.850169 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3848ec-417d-466f-ba32-a3228643eaf1 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:15.850454 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3848ec-417d-466f-ba32-a3228643eaf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:15.850732 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3848ec-417d-466f-ba32-a3228643eaf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:15.852396 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-cc3848ec-417d-466f-ba32-a3228643eaf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:15.859763 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:24:15.960466 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3848ec-417d-466f-ba32-a3228643eaf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:24:15.961898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3848ec-417d-466f-ba32-a3228643eaf1 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:24:15.979791 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 657/1322] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:24:15 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:24:19.351589 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c54b80-8293-439d-94b2-72db6463218c None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:19.352121 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c54b80-8293-439d-94b2-72db6463218c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:19.352282 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c54b80-8293-439d-94b2-72db6463218c None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:19.353557 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 666/1323] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:24:19.359034 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb525f-246f-4596-b7b8-4aa91d1e1283 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:19.359455 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb525f-246f-4596-b7b8-4aa91d1e1283 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:19.359758 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb525f-246f-4596-b7b8-4aa91d1e1283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:19.425459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-b9fb525f-246f-4596-b7b8-4aa91d1e1283 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:24:19.427006 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-b9fb525f-246f-4596-b7b8-4aa91d1e1283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:19.488831 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:24:19.528581 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 658/1324] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:24:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:24:19.566725 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c54b80-8293-439d-94b2-72db6463218c None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:24:19.568287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-81c54b80-8293-439d-94b2-72db6463218c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:19.683284 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c54b80-8293-439d-94b2-72db6463218c None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:24:19.696606 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:24:19.760680 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94025020-7edb-47ef-af6a-14a3aa808162 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:24:19.761341 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94025020-7edb-47ef-af6a-14a3aa808162 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:19.761626 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94025020-7edb-47ef-af6a-14a3aa808162 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:19.761908 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94025020-7edb-47ef-af6a-14a3aa808162 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:19.764410 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-94025020-7edb-47ef-af6a-14a3aa808162 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:19.906363 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94025020-7edb-47ef-af6a-14a3aa808162 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:24:19.909053 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94025020-7edb-47ef-af6a-14a3aa808162 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:24:19.930803 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 667/1325] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:24:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:24:22.271802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88afa2-f2a0-40eb-a37c-8c60caaf8fd9 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:22.272269 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88afa2-f2a0-40eb-a37c-8c60caaf8fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:22.272695 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88afa2-f2a0-40eb-a37c-8c60caaf8fd9 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:22.274324 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 659/1326] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:24:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:24:22.279930 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaab7a3-9b6f-4d12-8d26-0c232cbe3e52 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:22.280486 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaab7a3-9b6f-4d12-8d26-0c232cbe3e52 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:22.280486 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaab7a3-9b6f-4d12-8d26-0c232cbe3e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:22.329534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:24:22.348702 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-0aaab7a3-9b6f-4d12-8d26-0c232cbe3e52 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:24:22.349912 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-0aaab7a3-9b6f-4d12-8d26-0c232cbe3e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:22.450074 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 668/1327] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:24:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:24:22.485276 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d88afa2-f2a0-40eb-a37c-8c60caaf8fd9 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:24:22.486843 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-2d88afa2-f2a0-40eb-a37c-8c60caaf8fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:22.597370 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d88afa2-f2a0-40eb-a37c-8c60caaf8fd9 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:24:22.670102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036b292-e4dc-45e7-99ac-76a56ce45890 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:24:22.670734 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036b292-e4dc-45e7-99ac-76a56ce45890 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:22.671014 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036b292-e4dc-45e7-99ac-76a56ce45890 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:22.671290 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036b292-e4dc-45e7-99ac-76a56ce45890 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:22.673606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-a036b292-e4dc-45e7-99ac-76a56ce45890 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:22.690567 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:24:22.787181 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036b292-e4dc-45e7-99ac-76a56ce45890 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:24:22.788671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036b292-e4dc-45e7-99ac-76a56ce45890 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:24:22.809251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 660/1328] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:24:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:24:25.082969 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e899e9c-e2fa-4d99-a992-3a3b8dee1736 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:25.083498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e899e9c-e2fa-4d99-a992-3a3b8dee1736 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:25.083498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e899e9c-e2fa-4d99-a992-3a3b8dee1736 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:25.084941 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 669/1329] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:24:25.090124 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af683589-baef-46bc-8275-7d6e930b5747 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:25.090539 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af683589-baef-46bc-8275-7d6e930b5747 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:25.090693 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af683589-baef-46bc-8275-7d6e930b5747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:25.155889 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-af683589-baef-46bc-8275-7d6e930b5747 None None] MFA Rules not processed for user `66b6b05f6db5476b801ab332f119cbdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:24:25.157796 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-af683589-baef-46bc-8275-7d6e930b5747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:25.226710 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:24:25.259662 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 661/1330] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:24:25 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:24:25.298025 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e899e9c-e2fa-4d99-a992-3a3b8dee1736 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:24:25.299562 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-0e899e9c-e2fa-4d99-a992-3a3b8dee1736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:25.404601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e899e9c-e2fa-4d99-a992-3a3b8dee1736 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:24:25.424874 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:24:25.479053 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d1223e-2362-4f29-97d3-62d1bc7fb868 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:24:25.479699 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1223e-2362-4f29-97d3-62d1bc7fb868 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:25.479980 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1223e-2362-4f29-97d3-62d1bc7fb868 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:25.480281 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1223e-2362-4f29-97d3-62d1bc7fb868 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:25.481906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b7d1223e-2362-4f29-97d3-62d1bc7fb868 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:25.588914 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d1223e-2362-4f29-97d3-62d1bc7fb868 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:24:25.590278 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d1223e-2362-4f29-97d3-62d1bc7fb868 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:24:25.608277 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 670/1331] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:24:25 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:24:28.114343 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d792b8e-0104-47df-8bad-728f5a5b53e1 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:28.114860 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d792b8e-0104-47df-8bad-728f5a5b53e1 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:28.114860 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d792b8e-0104-47df-8bad-728f5a5b53e1 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:28.116215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 662/1332] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:24:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:24:28.121369 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767545c0-b603-4626-8427-246b9d492750 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:28.121791 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767545c0-b603-4626-8427-246b9d492750 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:28.121940 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767545c0-b603-4626-8427-246b9d492750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:28.184508 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:24:28.188116 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-767545c0-b603-4626-8427-246b9d492750 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:24:28.189860 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-767545c0-b603-4626-8427-246b9d492750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:28.286756 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 671/1333] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:24:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:24:28.322426 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d792b8e-0104-47df-8bad-728f5a5b53e1 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:24:28.323972 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-9d792b8e-0104-47df-8bad-728f5a5b53e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:28.429316 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d792b8e-0104-47df-8bad-728f5a5b53e1 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:24:28.502517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686983f8-7382-4b76-96c0-c537a53b23de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:24:28.503165 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686983f8-7382-4b76-96c0-c537a53b23de service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:28.503451 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686983f8-7382-4b76-96c0-c537a53b23de service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:28.503729 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686983f8-7382-4b76-96c0-c537a53b23de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:28.505427 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-686983f8-7382-4b76-96c0-c537a53b23de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:28.532769 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:24:28.617549 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686983f8-7382-4b76-96c0-c537a53b23de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:24:28.619014 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686983f8-7382-4b76-96c0-c537a53b23de service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:24:28.638649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 663/1334] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:24:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:24:30.917499 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2fbb4-1557-4343-9188-420278f49f21 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:30.918064 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2fbb4-1557-4343-9188-420278f49f21 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:30.918064 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2fbb4-1557-4343-9188-420278f49f21 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:30.919449 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 672/1335] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:24:30.925399 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fab3b2-e080-45e9-b8ce-72ffee33649c None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:30.925833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fab3b2-e080-45e9-b8ce-72ffee33649c None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:30.925952 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fab3b2-e080-45e9-b8ce-72ffee33649c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:30.989526 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-e1fab3b2-e080-45e9-b8ce-72ffee33649c None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:24:30.990711 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-e1fab3b2-e080-45e9-b8ce-72ffee33649c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:31.064852 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:24:31.087426 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 664/1336] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:24:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:24:31.127318 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2fbb4-1557-4343-9188-420278f49f21 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:24:31.128895 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-bdd2fbb4-1557-4343-9188-420278f49f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:31.231246 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2fbb4-1557-4343-9188-420278f49f21 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:24:31.262170 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:24:31.309499 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9ab993-af8e-44b0-9296-8e11c63cd01d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:24:31.310155 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ab993-af8e-44b0-9296-8e11c63cd01d service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:31.310443 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ab993-af8e-44b0-9296-8e11c63cd01d service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:31.310726 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9ab993-af8e-44b0-9296-8e11c63cd01d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:31.312415 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-0f9ab993-af8e-44b0-9296-8e11c63cd01d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:31.424995 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9ab993-af8e-44b0-9296-8e11c63cd01d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:24:31.426432 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9ab993-af8e-44b0-9296-8e11c63cd01d service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:24:31.444245 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 673/1337] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:24:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:24:33.975896 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1507f-4c78-4b93-a5c4-47d2bd260ad6 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:33.976460 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1507f-4c78-4b93-a5c4-47d2bd260ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:33.976460 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1507f-4c78-4b93-a5c4-47d2bd260ad6 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:33.977855 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 665/1338] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:24:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:24:33.983076 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feefb996-5854-49e8-9c09-c8784724807a None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:33.983454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feefb996-5854-49e8-9c09-c8784724807a None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:33.983649 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feefb996-5854-49e8-9c09-c8784724807a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:34.054321 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-feefb996-5854-49e8-9c09-c8784724807a None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:24:34.055521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-feefb996-5854-49e8-9c09-c8784724807a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:34.063681 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:24:34.149922 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 674/1339] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:24:33 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:24:34.186071 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd1507f-4c78-4b93-a5c4-47d2bd260ad6 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:24:34.187655 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-efd1507f-4c78-4b93-a5c4-47d2bd260ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:34.291804 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd1507f-4c78-4b93-a5c4-47d2bd260ad6 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:24:34.364439 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e7a84-da22-4924-a270-c38ce29a18b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:24:34.365094 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e7a84-da22-4924-a270-c38ce29a18b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:34.365378 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e7a84-da22-4924-a270-c38ce29a18b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:34.365653 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e7a84-da22-4924-a270-c38ce29a18b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:34.367266 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-158e7a84-da22-4924-a270-c38ce29a18b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:34.402787 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:24:34.476728 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e7a84-da22-4924-a270-c38ce29a18b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:24:34.478183 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e7a84-da22-4924-a270-c38ce29a18b7 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:24:34.496473 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 666/1340] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:24:34 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:24:38.830101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf76e07-8ef3-41a0-8804-5a4c2000b293 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:38.830688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf76e07-8ef3-41a0-8804-5a4c2000b293 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:38.830688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf76e07-8ef3-41a0-8804-5a4c2000b293 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:38.831962 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 675/1341] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:24:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:24:38.837308 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f63bc-4a82-4381-b25a-b23fc7b22d0b None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:38.837781 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f63bc-4a82-4381-b25a-b23fc7b22d0b None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:38.837878 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f63bc-4a82-4381-b25a-b23fc7b22d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:38.912235 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-fc1f63bc-4a82-4381-b25a-b23fc7b22d0b None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:24:38.913759 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-fc1f63bc-4a82-4381-b25a-b23fc7b22d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:39.004766 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:24:39.020204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 667/1342] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:24:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:24:39.057339 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf76e07-8ef3-41a0-8804-5a4c2000b293 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:24:39.059490 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-bcf76e07-8ef3-41a0-8804-5a4c2000b293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:39.156754 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf76e07-8ef3-41a0-8804-5a4c2000b293 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:24:39.160399 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f04b58e-af31-4fc7-953b-2bd99c34e4ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:24:39.161055 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04b58e-af31-4fc7-953b-2bd99c34e4ce service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:39.161343 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04b58e-af31-4fc7-953b-2bd99c34e4ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:39.161622 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04b58e-af31-4fc7-953b-2bd99c34e4ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:39.170740 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f04b58e-af31-4fc7-953b-2bd99c34e4ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:24:39.172260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f04b58e-af31-4fc7-953b-2bd99c34e4ce service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:24:39.182024 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 676/1343] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:24:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:24:42.030198 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ecc2a2-d343-4b92-b4da-8bfa26a30d1d None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:42.030617 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ecc2a2-d343-4b92-b4da-8bfa26a30d1d None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:42.030779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ecc2a2-d343-4b92-b4da-8bfa26a30d1d None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:42.032368 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 668/1344] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:24:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:24:42.037521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c64ab-ca9d-4d1d-a630-72ccfc8902ba None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:42.037934 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c64ab-ca9d-4d1d-a630-72ccfc8902ba None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:42.038086 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c64ab-ca9d-4d1d-a630-72ccfc8902ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:42.066158 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-575c64ab-ca9d-4d1d-a630-72ccfc8902ba None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:24:42.067341 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-575c64ab-ca9d-4d1d-a630-72ccfc8902ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:42.094427 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:24:42.145852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 677/1345] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:24:42 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:24:42.182529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ecc2a2-d343-4b92-b4da-8bfa26a30d1d None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:24:42.193985 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ecc2a2-d343-4b92-b4da-8bfa26a30d1d None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:24:42.197778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593abfb3-3d77-4e9d-8aa6-afc5277373f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:24:42.198425 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593abfb3-3d77-4e9d-8aa6-afc5277373f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:42.198709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593abfb3-3d77-4e9d-8aa6-afc5277373f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:42.198989 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593abfb3-3d77-4e9d-8aa6-afc5277373f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:42.214952 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593abfb3-3d77-4e9d-8aa6-afc5277373f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:24:42.216442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593abfb3-3d77-4e9d-8aa6-afc5277373f3 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:24:42.226962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 669/1346] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:24:42 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:24:46.626081 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282eb52b-bf99-47c0-a85f-65e0c2a21725 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:46.626641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282eb52b-bf99-47c0-a85f-65e0c2a21725 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:46.626641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282eb52b-bf99-47c0-a85f-65e0c2a21725 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:46.628008 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 678/1347] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:24:46.633235 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd19ff-9ac3-4c22-b9bc-e0f7d5dd89d7 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:46.633661 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd19ff-9ac3-4c22-b9bc-e0f7d5dd89d7 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:46.633818 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd19ff-9ac3-4c22-b9bc-e0f7d5dd89d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:46.651935 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-22bd19ff-9ac3-4c22-b9bc-e0f7d5dd89d7 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:24:46.653163 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-22bd19ff-9ac3-4c22-b9bc-e0f7d5dd89d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:46.661700 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 670/1348] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:24:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:24:46.699643 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282eb52b-bf99-47c0-a85f-65e0c2a21725 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:24:46.710531 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282eb52b-bf99-47c0-a85f-65e0c2a21725 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:24:46.714064 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f33e52-0916-4004-90af-1b61a2af67b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:24:46.714778 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f33e52-0916-4004-90af-1b61a2af67b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:46.715126 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f33e52-0916-4004-90af-1b61a2af67b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:46.715476 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f33e52-0916-4004-90af-1b61a2af67b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:46.725067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f33e52-0916-4004-90af-1b61a2af67b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:24:46.726686 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f33e52-0916-4004-90af-1b61a2af67b0 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:24:46.737233 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 679/1349] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:24:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:24:52.186454 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0a2c3-6c7a-487f-82d6-8fab25821550 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:52.187683 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0a2c3-6c7a-487f-82d6-8fab25821550 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:52.187683 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0a2c3-6c7a-487f-82d6-8fab25821550 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:52.189467 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 671/1350] 10.222.0.16 () {54 vars in 921 bytes} [Mon May 23 12:24:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:24:52.197465 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74ea5a-5c49-4fa2-8841-15744351d5d3 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:52.197915 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74ea5a-5c49-4fa2-8841-15744351d5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:52.198027 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74ea5a-5c49-4fa2-8841-15744351d5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:52.275915 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-1c74ea5a-5c49-4fa2-8841-15744351d5d3 None None] MFA Rules not processed for user `ebc0f137e0d24dd2a460689bc17a1b98`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:24:52.277267 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1c74ea5a-5c49-4fa2-8841-15744351d5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:52.357178 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 680/1351] 10.222.0.16 () {58 vars in 1018 bytes} [Mon May 23 12:24:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:24:56.646903 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=13 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:24:56.951476 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e82c30-0a8c-4eb9-a58a-1130c633e4c8 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:56.951671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e82c30-0a8c-4eb9-a58a-1130c633e4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:56.951942 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e82c30-0a8c-4eb9-a58a-1130c633e4c8 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:56.961670 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 672/1352] 10.222.0.16 () {54 vars in 921 bytes} [Mon May 23 12:24:56 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:24:56.969109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e2919-51f0-42ad-bf71-f4bdb37571ca None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:56.969593 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e2919-51f0-42ad-bf71-f4bdb37571ca None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:56.969675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e2919-51f0-42ad-bf71-f4bdb37571ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:56.989256 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-133e2919-51f0-42ad-bf71-f4bdb37571ca None None] MFA Rules not processed for user `ebc0f137e0d24dd2a460689bc17a1b98`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:24:56.990567 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-133e2919-51f0-42ad-bf71-f4bdb37571ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:57.000656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 681/1353] 10.222.0.16 () {58 vars in 1018 bytes} [Mon May 23 12:24:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:24:58.556506 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3616f-d27d-4579-a62d-9f2625b7767b None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:58.557061 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3616f-d27d-4579-a62d-9f2625b7767b None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:58.557061 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3616f-d27d-4579-a62d-9f2625b7767b None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:58.558427 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 673/1354] 10.222.0.16 () {54 vars in 921 bytes} [Mon May 23 12:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:24:58.563498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118d526-e7f8-4a07-a231-3a48b2b989d7 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:24:58.563933 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118d526-e7f8-4a07-a231-3a48b2b989d7 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:24:58.564044 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118d526-e7f8-4a07-a231-3a48b2b989d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:24:58.583180 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-f118d526-e7f8-4a07-a231-3a48b2b989d7 None None] MFA Rules not processed for user `ebc0f137e0d24dd2a460689bc17a1b98`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:24:58.584424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-f118d526-e7f8-4a07-a231-3a48b2b989d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:24:58.592948 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 682/1355] 10.222.0.16 () {58 vars in 1018 bytes} [Mon May 23 12:24:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:25:02.680418 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c540c1f-fc7e-460b-8841-4fa24660c595 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:02.680944 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c540c1f-fc7e-460b-8841-4fa24660c595 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:02.681126 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c540c1f-fc7e-460b-8841-4fa24660c595 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:02.682696 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 674/1356] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:25:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:25:02.688555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590f21f-40e4-410d-81c7-e7a546be9b51 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:02.689280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590f21f-40e4-410d-81c7-e7a546be9b51 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:02.689280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d590f21f-40e4-410d-81c7-e7a546be9b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:02.708442 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-d590f21f-40e4-410d-81c7-e7a546be9b51 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:25:02.709653 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d590f21f-40e4-410d-81c7-e7a546be9b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:25:02.726112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 683/1357] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:25:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:25:02.783045 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac58752-47be-4cc0-a68c-c4419d86f053 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:02.784229 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac58752-47be-4cc0-a68c-c4419d86f053 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:02.784948 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac58752-47be-4cc0-a68c-c4419d86f053 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:02.788687 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 675/1358] 10.222.0.16 () {54 vars in 961 bytes} [Mon May 23 12:25:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:25:02.797605 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fd7b4-3d95-4a62-8a66-4c8b5500f693 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:02.798618 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fd7b4-3d95-4a62-8a66-4c8b5500f693 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:02.799215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fd7b4-3d95-4a62-8a66-4c8b5500f693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:02.845035 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:25:02.859956 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-129fd7b4-3d95-4a62-8a66-4c8b5500f693 None None] MFA Rules not processed for user `b83535b443a84275a160cd84cb3ad5a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:25:02.861206 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-129fd7b4-3d95-4a62-8a66-4c8b5500f693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:25:02.945195 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 684/1359] 10.222.0.16 () {58 vars in 1058 bytes} [Mon May 23 12:25:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:25:02.953420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac58752-47be-4cc0-a68c-c4419d86f053 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:25:02.970043 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac58752-47be-4cc0-a68c-c4419d86f053 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:25:02.975428 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f030e8-dab3-4f2d-a473-0428cca2798f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:25:02.976064 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f030e8-dab3-4f2d-a473-0428cca2798f service nova] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:02.976401 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f030e8-dab3-4f2d-a473-0428cca2798f service nova] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:02.976686 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f030e8-dab3-4f2d-a473-0428cca2798f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:02.985124 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f030e8-dab3-4f2d-a473-0428cca2798f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:25:02.986732 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f030e8-dab3-4f2d-a473-0428cca2798f service nova] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:25:02.996130 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 676/1360] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:25:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:25:05.431568 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5fcd2-daf2-4919-b113-78b846e6a9a4 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:05.432096 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5fcd2-daf2-4919-b113-78b846e6a9a4 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:05.432357 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5fcd2-daf2-4919-b113-78b846e6a9a4 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:05.433670 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 685/1361] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:25:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:25:05.439439 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f207c6-7434-4b5d-b1ed-e2a83073f46a None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:05.439886 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f207c6-7434-4b5d-b1ed-e2a83073f46a None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:05.440231 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f207c6-7434-4b5d-b1ed-e2a83073f46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:05.459283 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-b8f207c6-7434-4b5d-b1ed-e2a83073f46a None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:25:05.460886 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-b8f207c6-7434-4b5d-b1ed-e2a83073f46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:25:05.470333 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 677/1362] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:25:05.513881 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5fcd2-daf2-4919-b113-78b846e6a9a4 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:25:05.528745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5fcd2-daf2-4919-b113-78b846e6a9a4 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:25:05.535051 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8554dbc-60cd-4a46-a21a-ea0d6ec7cfcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:25:05.535691 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8554dbc-60cd-4a46-a21a-ea0d6ec7cfcd service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:05.535978 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8554dbc-60cd-4a46-a21a-ea0d6ec7cfcd service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:05.536290 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8554dbc-60cd-4a46-a21a-ea0d6ec7cfcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:05.544917 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8554dbc-60cd-4a46-a21a-ea0d6ec7cfcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:25:05.547154 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8554dbc-60cd-4a46-a21a-ea0d6ec7cfcd service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:25:05.557198 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 686/1363] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:25:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:25:08.168319 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f8bf2-8fd6-4e3e-82fb-6950e19f68c4 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:08.168828 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f8bf2-8fd6-4e3e-82fb-6950e19f68c4 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:08.168926 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f8bf2-8fd6-4e3e-82fb-6950e19f68c4 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:08.170332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 678/1364] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:25:08.177002 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8099b5-bd16-46d7-987c-62d337d50a72 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:08.177563 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8099b5-bd16-46d7-987c-62d337d50a72 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:08.177563 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8099b5-bd16-46d7-987c-62d337d50a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:08.196733 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-8a8099b5-bd16-46d7-987c-62d337d50a72 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:25:08.197914 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-8a8099b5-bd16-46d7-987c-62d337d50a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:25:08.206192 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 687/1365] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:25:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:25:08.253017 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcbf93-77d4-4153-92b4-62e8a5bcebff None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:08.253312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcbf93-77d4-4153-92b4-62e8a5bcebff None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:08.253595 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcbf93-77d4-4153-92b4-62e8a5bcebff None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:08.255068 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 679/1366] 10.222.0.16 () {54 vars in 961 bytes} [Mon May 23 12:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:25:08.260391 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf60cb-4e78-4db3-aeaf-9d7be0f22cdf None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:08.260780 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf60cb-4e78-4db3-aeaf-9d7be0f22cdf None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:08.261117 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf60cb-4e78-4db3-aeaf-9d7be0f22cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:08.281187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-8bbf60cb-4e78-4db3-aeaf-9d7be0f22cdf None None] MFA Rules not processed for user `b83535b443a84275a160cd84cb3ad5a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:25:08.282303 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-8bbf60cb-4e78-4db3-aeaf-9d7be0f22cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:25:08.292685 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 688/1367] 10.222.0.16 () {58 vars in 1058 bytes} [Mon May 23 12:25:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:25:08.301546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dcbf93-77d4-4153-92b4-62e8a5bcebff None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:25:08.325313 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dcbf93-77d4-4153-92b4-62e8a5bcebff None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:25:08.337058 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ef085-d1b8-41a5-b91e-e00567d2b555 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:25:08.337731 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ef085-d1b8-41a5-b91e-e00567d2b555 service nova] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:08.338118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ef085-d1b8-41a5-b91e-e00567d2b555 service nova] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:08.338405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ef085-d1b8-41a5-b91e-e00567d2b555 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:08.350444 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ef085-d1b8-41a5-b91e-e00567d2b555 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:25:08.352152 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ef085-d1b8-41a5-b91e-e00567d2b555 service nova] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:25:08.362723 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 680/1368] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:25:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:25:10.702829 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf28afb-e35c-4418-8e37-6906a2b8b68c None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:10.703475 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf28afb-e35c-4418-8e37-6906a2b8b68c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:10.703475 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf28afb-e35c-4418-8e37-6906a2b8b68c None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:10.704797 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 689/1369] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:25:10.710311 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca00126-ce77-49c4-8d7e-d5c378d0e03b None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:10.711151 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca00126-ce77-49c4-8d7e-d5c378d0e03b None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:10.711443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca00126-ce77-49c4-8d7e-d5c378d0e03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:10.731106 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-8ca00126-ce77-49c4-8d7e-d5c378d0e03b None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:25:10.732359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-8ca00126-ce77-49c4-8d7e-d5c378d0e03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:25:10.744938 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 681/1370] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:25:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:25:10.785567 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf28afb-e35c-4418-8e37-6906a2b8b68c None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:25:10.801637 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf28afb-e35c-4418-8e37-6906a2b8b68c None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:25:10.807452 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b536d7a-f4a7-48ca-8044-cbef2a7c7257 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:25:10.808261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b536d7a-f4a7-48ca-8044-cbef2a7c7257 service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:10.808718 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b536d7a-f4a7-48ca-8044-cbef2a7c7257 service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:10.809085 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b536d7a-f4a7-48ca-8044-cbef2a7c7257 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:10.818222 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b536d7a-f4a7-48ca-8044-cbef2a7c7257 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:25:10.819912 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b536d7a-f4a7-48ca-8044-cbef2a7c7257 service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:25:10.830465 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 690/1371] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:25:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:25:11.122338 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c96968-6788-4d2d-bdc6-eded0743c905 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:11.122623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c96968-6788-4d2d-bdc6-eded0743c905 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:11.123095 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c96968-6788-4d2d-bdc6-eded0743c905 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:11.125173 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 682/1372] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:25:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:25:11.130706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c752837-a5e9-4162-a350-f50f06a7ab2d None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:11.130998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c752837-a5e9-4162-a350-f50f06a7ab2d None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:11.131273 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c752837-a5e9-4162-a350-f50f06a7ab2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:11.150116 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-8c752837-a5e9-4162-a350-f50f06a7ab2d None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:25:11.151312 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-8c752837-a5e9-4162-a350-f50f06a7ab2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:25:11.159790 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 691/1373] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:25:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:25:11.469267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6fd26-b997-4919-a617-42f364e8b4f7 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:11.469677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6fd26-b997-4919-a617-42f364e8b4f7 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:11.470059 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6fd26-b997-4919-a617-42f364e8b4f7 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:11.471602 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 683/1374] 10.222.0.16 () {54 vars in 963 bytes} [Mon May 23 12:25:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:25:11.478004 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b097653-66ee-49b4-a29d-1782a98a9adc None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:11.478406 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b097653-66ee-49b4-a29d-1782a98a9adc None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:11.479007 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b097653-66ee-49b4-a29d-1782a98a9adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:11.532420 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-9b097653-66ee-49b4-a29d-1782a98a9adc None None] MFA Rules not processed for user `e12ef3e115544a51a6bf9b08d265fefb`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:25:11.533754 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-9b097653-66ee-49b4-a29d-1782a98a9adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:25:11.600491 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 692/1375] 10.222.0.16 () {58 vars in 1060 bytes} [Mon May 23 12:25:11 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:25:11.608140 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6fd26-b997-4919-a617-42f364e8b4f7 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:25:11.620613 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6fd26-b997-4919-a617-42f364e8b4f7 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:25:11.626801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c2067-b1a4-4c17-9c66-8f0849349f16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:25:11.627459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c2067-b1a4-4c17-9c66-8f0849349f16 service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:11.627750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c2067-b1a4-4c17-9c66-8f0849349f16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:11.628034 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c2067-b1a4-4c17-9c66-8f0849349f16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:11.638575 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c2067-b1a4-4c17-9c66-8f0849349f16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:25:11.641125 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c2067-b1a4-4c17-9c66-8f0849349f16 service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:25:11.652547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 684/1376] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:25:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:25:13.120676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d697b3a-c11f-4925-8bc3-0009a54dff86 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:13.121219 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d697b3a-c11f-4925-8bc3-0009a54dff86 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:13.121219 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d697b3a-c11f-4925-8bc3-0009a54dff86 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:13.122611 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 693/1377] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:25:13.128310 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674fc28-6478-4e40-ada6-ebeeaeb31df9 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:13.128740 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674fc28-6478-4e40-ada6-ebeeaeb31df9 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:13.128854 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674fc28-6478-4e40-ada6-ebeeaeb31df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:13.147306 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-e674fc28-6478-4e40-ada6-ebeeaeb31df9 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:25:13.148609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-e674fc28-6478-4e40-ada6-ebeeaeb31df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:25:13.156520 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 685/1378] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:25:13.199931 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d697b3a-c11f-4925-8bc3-0009a54dff86 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:25:13.216400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d697b3a-c11f-4925-8bc3-0009a54dff86 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:25:13.231832 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683a3822-1ba9-4156-b503-ea6bce0d7156 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:25:13.232566 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a3822-1ba9-4156-b503-ea6bce0d7156 service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:13.232848 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a3822-1ba9-4156-b503-ea6bce0d7156 service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:13.233128 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a3822-1ba9-4156-b503-ea6bce0d7156 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:13.243474 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683a3822-1ba9-4156-b503-ea6bce0d7156 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:25:13.246735 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683a3822-1ba9-4156-b503-ea6bce0d7156 service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:25:13.254507 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:25:13.261957 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 694/1379] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:25:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:25:15.554054 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd74f7f-ce52-4c78-bf94-c915b709b568 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:15.554573 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd74f7f-ce52-4c78-bf94-c915b709b568 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:15.554573 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd74f7f-ce52-4c78-bf94-c915b709b568 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:15.555813 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 686/1380] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:25:15.561552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495dbe0e-e4ae-4c06-86fd-5f7b28dce29d None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:15.562251 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495dbe0e-e4ae-4c06-86fd-5f7b28dce29d None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:15.562251 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495dbe0e-e4ae-4c06-86fd-5f7b28dce29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:15.582051 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-495dbe0e-e4ae-4c06-86fd-5f7b28dce29d None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:25:15.583235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-495dbe0e-e4ae-4c06-86fd-5f7b28dce29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:25:15.591341 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 695/1381] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:25:15.635507 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd74f7f-ce52-4c78-bf94-c915b709b568 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:25:15.651556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd74f7f-ce52-4c78-bf94-c915b709b568 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:25:15.657815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26c263-279a-4fc5-bf57-0530cb889e05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:25:15.658576 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26c263-279a-4fc5-bf57-0530cb889e05 service nova] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:15.658964 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26c263-279a-4fc5-bf57-0530cb889e05 service nova] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:15.659344 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26c263-279a-4fc5-bf57-0530cb889e05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:15.668376 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b26c263-279a-4fc5-bf57-0530cb889e05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:25:15.670087 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b26c263-279a-4fc5-bf57-0530cb889e05 service nova] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:25:15.681176 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 687/1382] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:25:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:25:17.970511 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5a1bd-4728-4808-8535-61a23e4db77c None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:17.971044 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5a1bd-4728-4808-8535-61a23e4db77c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:17.971310 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5a1bd-4728-4808-8535-61a23e4db77c None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:17.972819 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 696/1383] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:25:17.978059 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483480c8-0b90-4152-a81b-5992905ef89a None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:17.978476 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483480c8-0b90-4152-a81b-5992905ef89a None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:17.978632 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483480c8-0b90-4152-a81b-5992905ef89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:17.996839 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-483480c8-0b90-4152-a81b-5992905ef89a None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:25:17.998018 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-483480c8-0b90-4152-a81b-5992905ef89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:25:18.006362 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 688/1384] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:25:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:25:18.049718 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5a1bd-4728-4808-8535-61a23e4db77c None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:25:18.065249 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5a1bd-4728-4808-8535-61a23e4db77c None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:25:18.070920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5314f4b-70fc-4814-a4b8-fc2b41ea2fb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:25:18.071500 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5314f4b-70fc-4814-a4b8-fc2b41ea2fb4 service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:18.071956 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5314f4b-70fc-4814-a4b8-fc2b41ea2fb4 service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:18.072264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5314f4b-70fc-4814-a4b8-fc2b41ea2fb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:18.081948 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5314f4b-70fc-4814-a4b8-fc2b41ea2fb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:25:18.083512 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5314f4b-70fc-4814-a4b8-fc2b41ea2fb4 service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:25:18.097550 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 697/1385] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:25:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:25:20.370391 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d9bc8-d3ba-40a3-9dd2-81c965e7e10c None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:20.371176 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d9bc8-d3ba-40a3-9dd2-81c965e7e10c None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:20.371176 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d9bc8-d3ba-40a3-9dd2-81c965e7e10c None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:20.372306 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 689/1386] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:25:20.378034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fee2f6-7458-467d-a2a5-c05bcfec30cb None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:20.378528 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fee2f6-7458-467d-a2a5-c05bcfec30cb None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:20.378686 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fee2f6-7458-467d-a2a5-c05bcfec30cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:20.396371 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a8fee2f6-7458-467d-a2a5-c05bcfec30cb None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:25:20.397551 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a8fee2f6-7458-467d-a2a5-c05bcfec30cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:25:20.405492 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 698/1387] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:25:20.449623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d9bc8-d3ba-40a3-9dd2-81c965e7e10c None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:25:20.464133 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d9bc8-d3ba-40a3-9dd2-81c965e7e10c None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:25:20.469627 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a0e288-917f-490e-8e4c-b19054a2a890 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:25:20.470262 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0e288-917f-490e-8e4c-b19054a2a890 service nova] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:20.470547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0e288-917f-490e-8e4c-b19054a2a890 service nova] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:20.470907 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a0e288-917f-490e-8e4c-b19054a2a890 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:20.479145 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a0e288-917f-490e-8e4c-b19054a2a890 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:25:20.480704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a0e288-917f-490e-8e4c-b19054a2a890 service nova] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:25:20.490220 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 690/1388] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:25:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:25:22.724025 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b9456-075d-4f3c-acfa-bf82da15be75 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:22.725072 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b9456-075d-4f3c-acfa-bf82da15be75 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:22.725072 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b9456-075d-4f3c-acfa-bf82da15be75 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:22.725755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 699/1389] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:25:22.730970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f6803-d87a-4ff6-9c75-40864ab740ee None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:22.731443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f6803-d87a-4ff6-9c75-40864ab740ee None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:22.731591 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f6803-d87a-4ff6-9c75-40864ab740ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:22.750288 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-bd6f6803-d87a-4ff6-9c75-40864ab740ee None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:25:22.751483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-bd6f6803-d87a-4ff6-9c75-40864ab740ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:25:22.759668 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 691/1390] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:25:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:25:22.804264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b9456-075d-4f3c-acfa-bf82da15be75 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:25:22.822998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b9456-075d-4f3c-acfa-bf82da15be75 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:25:22.830082 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd5c787-a210-435d-9949-999aaeedb7f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:25:22.830942 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5c787-a210-435d-9949-999aaeedb7f2 service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:22.831318 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5c787-a210-435d-9949-999aaeedb7f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:22.831651 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5c787-a210-435d-9949-999aaeedb7f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:22.842089 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd5c787-a210-435d-9949-999aaeedb7f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:25:22.844912 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd5c787-a210-435d-9949-999aaeedb7f2 service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:25:22.856918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 700/1391] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:25:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:25:25.111400 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493d457-cbf3-4645-b8d0-be1f3dbe401b None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:25.112107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493d457-cbf3-4645-b8d0-be1f3dbe401b None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:25.112107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493d457-cbf3-4645-b8d0-be1f3dbe401b None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:25.113343 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 692/1392] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:25:25.118660 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545cecaa-6bb5-4966-8cad-3a7bc63a587c None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:25.119205 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545cecaa-6bb5-4966-8cad-3a7bc63a587c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:25.119320 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545cecaa-6bb5-4966-8cad-3a7bc63a587c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:25.140749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-545cecaa-6bb5-4966-8cad-3a7bc63a587c None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:25:25.141985 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-545cecaa-6bb5-4966-8cad-3a7bc63a587c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:25:25.150009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 701/1393] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:25:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:25:25.204547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493d457-cbf3-4645-b8d0-be1f3dbe401b None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:25:25.219990 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493d457-cbf3-4645-b8d0-be1f3dbe401b None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:25:25.225705 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb2629-8393-4107-b611-b72f25c37222 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:25:25.226295 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb2629-8393-4107-b611-b72f25c37222 service nova] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:25.226652 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb2629-8393-4107-b611-b72f25c37222 service nova] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:25.226916 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb2629-8393-4107-b611-b72f25c37222 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:25.237981 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb2629-8393-4107-b611-b72f25c37222 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:25:25.240050 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb2629-8393-4107-b611-b72f25c37222 service nova] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:25:25.252968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 693/1394] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:25:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:25:27.547292 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1036c-b83e-4dcc-90bb-c23de4d2db8f None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:27.547923 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1036c-b83e-4dcc-90bb-c23de4d2db8f None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:27.547923 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c1036c-b83e-4dcc-90bb-c23de4d2db8f None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:27.549359 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 702/1395] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:25:27.555478 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ff324-b278-4bf3-a73d-81c4a5337063 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:27.556058 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ff324-b278-4bf3-a73d-81c4a5337063 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:27.556058 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ff324-b278-4bf3-a73d-81c4a5337063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:27.562567 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=52 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:25:27.577176 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-7e5ff324-b278-4bf3-a73d-81c4a5337063 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:25:27.578593 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-7e5ff324-b278-4bf3-a73d-81c4a5337063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:25:27.586629 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 694/1396] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:25:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:25:27.631356 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1036c-b83e-4dcc-90bb-c23de4d2db8f None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:25:27.647849 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c1036c-b83e-4dcc-90bb-c23de4d2db8f None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:25:27.653289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d7adc-bad2-48bf-a7ef-e05d6e90cf99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:25:27.653927 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d7adc-bad2-48bf-a7ef-e05d6e90cf99 service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:27.654214 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d7adc-bad2-48bf-a7ef-e05d6e90cf99 service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:27.654492 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d7adc-bad2-48bf-a7ef-e05d6e90cf99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:27.663394 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964d7adc-bad2-48bf-a7ef-e05d6e90cf99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:25:27.665082 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964d7adc-bad2-48bf-a7ef-e05d6e90cf99 service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:25:27.674525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 703/1397] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:25:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:25:27.744417 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f849f-4f98-4ebc-8664-bde34496d0dd None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:27.744417 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f849f-4f98-4ebc-8664-bde34496d0dd None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:27.744417 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f849f-4f98-4ebc-8664-bde34496d0dd None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:27.744417 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 695/1398] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:25:27.745036 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a852e8da-f0ac-47e4-9434-8d906796d31a None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:27.745338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a852e8da-f0ac-47e4-9434-8d906796d31a None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:27.745625 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a852e8da-f0ac-47e4-9434-8d906796d31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:27.765735 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a852e8da-f0ac-47e4-9434-8d906796d31a None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:25:27.766988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a852e8da-f0ac-47e4-9434-8d906796d31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:25:27.775200 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 704/1399] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:25:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:25:27.839444 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c0eed-5052-4733-8686-f743b9eb08a4 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:27.839732 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c0eed-5052-4733-8686-f743b9eb08a4 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:27.840014 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c0eed-5052-4733-8686-f743b9eb08a4 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:27.841414 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 696/1400] 10.222.0.16 () {54 vars in 962 bytes} [Mon May 23 12:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:25:27.847606 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b39c2a-664b-4795-8ae3-65e8bc03fcd5 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:27.847899 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b39c2a-664b-4795-8ae3-65e8bc03fcd5 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:27.848249 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b39c2a-664b-4795-8ae3-65e8bc03fcd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:27.908435 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-70b39c2a-664b-4795-8ae3-65e8bc03fcd5 None None] MFA Rules not processed for user `2483a351f5e943ad8c872a98a4717b0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:25:27.909774 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-70b39c2a-664b-4795-8ae3-65e8bc03fcd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:25:27.971188 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 705/1401] 10.222.0.16 () {58 vars in 1059 bytes} [Mon May 23 12:25:27 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:25:27.978735 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c0eed-5052-4733-8686-f743b9eb08a4 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:25:27.989726 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c0eed-5052-4733-8686-f743b9eb08a4 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:25:27.993477 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9c7cb4-f34b-42f0-8d95-7602a641da95 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2483a351f5e943ad8c872a98a4717b0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:25:27.994123 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9c7cb4-f34b-42f0-8d95-7602a641da95 service glance] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:27.994412 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9c7cb4-f34b-42f0-8d95-7602a641da95 service glance] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:27.994698 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9c7cb4-f34b-42f0-8d95-7602a641da95 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:28.007956 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9c7cb4-f34b-42f0-8d95-7602a641da95 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:25:28.009446 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9c7cb4-f34b-42f0-8d95-7602a641da95 service glance] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:25:28.019999 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 697/1402] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:25:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:25:28.574824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01cfae-2447-46ed-aaf3-7e37da7650be None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:28.575326 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01cfae-2447-46ed-aaf3-7e37da7650be None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:28.575506 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01cfae-2447-46ed-aaf3-7e37da7650be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:28.624389 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-4b01cfae-2447-46ed-aaf3-7e37da7650be None None] MFA Rules not processed for user `733f8296ec3d4795a1140c1e1cb6eb95`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:25:28.625930 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-4b01cfae-2447-46ed-aaf3-7e37da7650be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:25:28.669409 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:25:28.693178 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 706/1403] 10.222.0.16 () {58 vars in 1009 bytes} [Mon May 23 12:25:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:25:28.738882 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaceef42-2045-4ba6-b5ea-5b86e71049ec None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:28.739730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaceef42-2045-4ba6-b5ea-5b86e71049ec None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:28.740016 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaceef42-2045-4ba6-b5ea-5b86e71049ec None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:28.741464 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 698/1404] 10.222.0.16 () {54 vars in 942 bytes} [Mon May 23 12:25:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:25:28.748522 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47edbe5-8ec9-4d2d-b16b-cb76abd7e443 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:28.749019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47edbe5-8ec9-4d2d-b16b-cb76abd7e443 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:28.749019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47edbe5-8ec9-4d2d-b16b-cb76abd7e443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:28.811904 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-f47edbe5-8ec9-4d2d-b16b-cb76abd7e443 None None] MFA Rules not processed for user `71fb9a79ac494635804b93d4a0048205`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:25:28.813202 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-f47edbe5-8ec9-4d2d-b16b-cb76abd7e443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:25:28.887342 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 707/1405] 10.222.0.16 () {58 vars in 1039 bytes} [Mon May 23 12:25:28 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:25:28.897521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaceef42-2045-4ba6-b5ea-5b86e71049ec None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:25:28.913029 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaceef42-2045-4ba6-b5ea-5b86e71049ec None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:25:28.917941 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84175b5c-b7a0-4dd8-a27b-56bf4cf1b657 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71fb9a79ac494635804b93d4a0048205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:25:28.918652 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84175b5c-b7a0-4dd8-a27b-56bf4cf1b657 service swift] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:28.919024 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84175b5c-b7a0-4dd8-a27b-56bf4cf1b657 service swift] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:28.919367 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84175b5c-b7a0-4dd8-a27b-56bf4cf1b657 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:28.927731 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84175b5c-b7a0-4dd8-a27b-56bf4cf1b657 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:25:28.929351 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84175b5c-b7a0-4dd8-a27b-56bf4cf1b657 service swift] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:25:28.938155 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 699/1406] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 12:25:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 12:25:29.962126 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e926150-9d29-4541-82fa-059f53c1eb22 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:29.962648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e926150-9d29-4541-82fa-059f53c1eb22 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:29.962648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e926150-9d29-4541-82fa-059f53c1eb22 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:29.964116 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 708/1407] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:25:29.969230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c548d64-2532-4a12-ab92-2746954fcb1a None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:29.969700 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c548d64-2532-4a12-ab92-2746954fcb1a None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:29.969998 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c548d64-2532-4a12-ab92-2746954fcb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:29.991592 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-9c548d64-2532-4a12-ab92-2746954fcb1a None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:25:29.993131 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-9c548d64-2532-4a12-ab92-2746954fcb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:25:30.001725 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 700/1408] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:25:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:25:30.047163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e926150-9d29-4541-82fa-059f53c1eb22 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:25:30.061987 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e926150-9d29-4541-82fa-059f53c1eb22 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:25:30.067524 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ec54e-5208-4c66-a3d9-0373d19d6323 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:25:30.068325 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ec54e-5208-4c66-a3d9-0373d19d6323 service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:30.068677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ec54e-5208-4c66-a3d9-0373d19d6323 service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:30.069032 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ec54e-5208-4c66-a3d9-0373d19d6323 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:30.078559 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ec54e-5208-4c66-a3d9-0373d19d6323 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:25:30.080280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ec54e-5208-4c66-a3d9-0373d19d6323 service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:25:30.090149 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 709/1409] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:25:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:25:39.983110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=17 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:25:40.085791 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=59 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:25:53.034676 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb023ba0-ba83-474c-9d95-eeb42d9b3e0b None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:53.035273 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb023ba0-ba83-474c-9d95-eeb42d9b3e0b None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:53.035273 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb023ba0-ba83-474c-9d95-eeb42d9b3e0b None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:53.036726 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 701/1410] 10.222.0.16 () {54 vars in 918 bytes} [Mon May 23 12:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:25:53.041938 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fc8e1-746b-43b7-89e0-5640fe152b93 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:53.042425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fc8e1-746b-43b7-89e0-5640fe152b93 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:53.042533 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fc8e1-746b-43b7-89e0-5640fe152b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:53.071943 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-8c3fc8e1-746b-43b7-89e0-5640fe152b93 None None] MFA Rules not processed for user `ebc0f137e0d24dd2a460689bc17a1b98`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:25:53.073960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-8c3fc8e1-746b-43b7-89e0-5640fe152b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:25:53.086772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 710/1411] 10.222.0.16 () {58 vars in 1015 bytes} [Mon May 23 12:25:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:25:53.101503 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e1dd67-f9b3-46be-b1f4-700e0b37354d None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:53.101794 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e1dd67-f9b3-46be-b1f4-700e0b37354d None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:53.102076 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e1dd67-f9b3-46be-b1f4-700e0b37354d None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:53.103426 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 702/1412] 10.222.0.16 () {54 vars in 918 bytes} [Mon May 23 12:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:25:53.108848 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21188c-0d62-46d8-a5b2-aa6d47919ac0 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:53.109200 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21188c-0d62-46d8-a5b2-aa6d47919ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:53.109487 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21188c-0d62-46d8-a5b2-aa6d47919ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:53.128197 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-6b21188c-0d62-46d8-a5b2-aa6d47919ac0 None None] MFA Rules not processed for user `b83535b443a84275a160cd84cb3ad5a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:25:53.129409 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-6b21188c-0d62-46d8-a5b2-aa6d47919ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:25:53.138490 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 711/1413] 10.222.0.16 () {58 vars in 1015 bytes} [Mon May 23 12:25:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:25:55.898583 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62817529-bf92-4f69-8afa-be3d3df2c140 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:55.899131 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62817529-bf92-4f69-8afa-be3d3df2c140 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:55.899131 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62817529-bf92-4f69-8afa-be3d3df2c140 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:55.900520 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 703/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 12:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:25:55.906036 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77f771-d7a6-49d5-8f9e-49cc2a2ddabb None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:55.906481 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77f771-d7a6-49d5-8f9e-49cc2a2ddabb None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:55.906575 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f77f771-d7a6-49d5-8f9e-49cc2a2ddabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:55.924108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-4f77f771-d7a6-49d5-8f9e-49cc2a2ddabb None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:25:55.925332 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-4f77f771-d7a6-49d5-8f9e-49cc2a2ddabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:25:55.933016 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 712/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 12:25:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:25:55.976700 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62817529-bf92-4f69-8afa-be3d3df2c140 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:25:55.992761 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62817529-bf92-4f69-8afa-be3d3df2c140 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:25:55.998221 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452d609-71dc-4783-bcce-eeb16a8b5dc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:25:55.999469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452d609-71dc-4783-bcce-eeb16a8b5dc3 service nova] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:55.999757 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452d609-71dc-4783-bcce-eeb16a8b5dc3 service nova] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:56.000058 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452d609-71dc-4783-bcce-eeb16a8b5dc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:56.008389 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a452d609-71dc-4783-bcce-eeb16a8b5dc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:25:56.010067 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a452d609-71dc-4783-bcce-eeb16a8b5dc3 service nova] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:25:56.019755 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 704/1416] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:25:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:25:57.017244 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5c336-1b66-4c8e-88f0-8c45af6937fe None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:57.017819 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5c336-1b66-4c8e-88f0-8c45af6937fe None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:57.017819 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5c336-1b66-4c8e-88f0-8c45af6937fe None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:57.019258 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 713/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 12:25:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:25:57.024486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbf93d-e9a3-4973-9a57-671df13b0cc9 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:57.024940 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbf93d-e9a3-4973-9a57-671df13b0cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:57.025045 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbf93d-e9a3-4973-9a57-671df13b0cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:57.043671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-2ccbf93d-e9a3-4973-9a57-671df13b0cc9 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:25:57.044896 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-2ccbf93d-e9a3-4973-9a57-671df13b0cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:25:57.052818 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 705/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 12:25:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:25:57.251497 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c5c336-1b66-4c8e-88f0-8c45af6937fe None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:25:57.265754 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c5c336-1b66-4c8e-88f0-8c45af6937fe None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:25:57.271160 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e977a561-bf5b-44f2-80f8-ef3160429930 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:25:57.271732 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977a561-bf5b-44f2-80f8-ef3160429930 service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:57.272015 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977a561-bf5b-44f2-80f8-ef3160429930 service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:57.272332 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977a561-bf5b-44f2-80f8-ef3160429930 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:57.280919 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e977a561-bf5b-44f2-80f8-ef3160429930 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:25:57.282509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e977a561-bf5b-44f2-80f8-ef3160429930 service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:25:57.292239 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 714/1419] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:25:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:25:58.238960 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e9adb-2d72-4c8b-8bac-d9ad6be93093 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:58.239490 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e9adb-2d72-4c8b-8bac-d9ad6be93093 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:58.239490 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e9adb-2d72-4c8b-8bac-d9ad6be93093 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:58.240801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 706/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 12:25:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:25:58.246000 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a2f74-a265-4eb2-b8ac-676f97061eda None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:58.246419 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a2f74-a265-4eb2-b8ac-676f97061eda None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:58.246573 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a2f74-a265-4eb2-b8ac-676f97061eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:58.265291 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-4f3a2f74-a265-4eb2-b8ac-676f97061eda None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:25:58.266498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-4f3a2f74-a265-4eb2-b8ac-676f97061eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:25:58.274316 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 715/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 12:25:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:25:58.425187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e9adb-2d72-4c8b-8bac-d9ad6be93093 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:25:58.439846 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e9adb-2d72-4c8b-8bac-d9ad6be93093 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:25:58.445249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cd3b4f-859b-4f68-9ce4-8e6d76953953 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:25:58.446144 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd3b4f-859b-4f68-9ce4-8e6d76953953 service nova] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:58.446430 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd3b4f-859b-4f68-9ce4-8e6d76953953 service nova] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:58.446711 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd3b4f-859b-4f68-9ce4-8e6d76953953 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:58.455792 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cd3b4f-859b-4f68-9ce4-8e6d76953953 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:25:58.457444 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cd3b4f-859b-4f68-9ce4-8e6d76953953 service nova] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:25:58.467005 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 707/1422] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:25:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:25:59.396755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903735f0-dfa6-40d9-83b9-32fdaadd0b96 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:59.397333 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903735f0-dfa6-40d9-83b9-32fdaadd0b96 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:59.397333 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903735f0-dfa6-40d9-83b9-32fdaadd0b96 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:59.398687 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 716/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 12:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:25:59.403727 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937a69e-882d-4ac7-a14a-105786713c44 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:59.404149 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937a69e-882d-4ac7-a14a-105786713c44 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:59.404322 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937a69e-882d-4ac7-a14a-105786713c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:59.422268 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-0937a69e-882d-4ac7-a14a-105786713c44 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:25:59.423482 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-0937a69e-882d-4ac7-a14a-105786713c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:25:59.431387 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 708/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 12:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:25:59.583161 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903735f0-dfa6-40d9-83b9-32fdaadd0b96 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:25:59.597745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903735f0-dfa6-40d9-83b9-32fdaadd0b96 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:25:59.603031 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b27a4b-7936-4f9c-ab30-4832c4f034d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:25:59.603668 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b27a4b-7936-4f9c-ab30-4832c4f034d6 service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:25:59.603953 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b27a4b-7936-4f9c-ab30-4832c4f034d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:25:59.604272 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b27a4b-7936-4f9c-ab30-4832c4f034d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:25:59.613098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b27a4b-7936-4f9c-ab30-4832c4f034d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:25:59.614676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b27a4b-7936-4f9c-ab30-4832c4f034d6 service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:25:59.624027 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 717/1425] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:25:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:26:00.556104 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2cda6-46b0-4a46-a6b4-7eac99e6a1b8 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:00.556711 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2cda6-46b0-4a46-a6b4-7eac99e6a1b8 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:00.556711 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2cda6-46b0-4a46-a6b4-7eac99e6a1b8 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:00.558119 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 709/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 12:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:26:00.563362 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222ff00-ae14-4c84-94a2-3a2c735844db None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:00.563838 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222ff00-ae14-4c84-94a2-3a2c735844db None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:00.563961 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222ff00-ae14-4c84-94a2-3a2c735844db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:00.582664 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-7222ff00-ae14-4c84-94a2-3a2c735844db None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:26:00.583874 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-7222ff00-ae14-4c84-94a2-3a2c735844db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:26:00.591645 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 718/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 12:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:26:00.748609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2cda6-46b0-4a46-a6b4-7eac99e6a1b8 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:26:00.763258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2cda6-46b0-4a46-a6b4-7eac99e6a1b8 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:26:00.774122 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687afb03-f219-4d3d-a46f-1c40a4b2f547 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:26:00.774780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687afb03-f219-4d3d-a46f-1c40a4b2f547 service nova] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:00.775071 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687afb03-f219-4d3d-a46f-1c40a4b2f547 service nova] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:00.775356 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687afb03-f219-4d3d-a46f-1c40a4b2f547 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:00.785176 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687afb03-f219-4d3d-a46f-1c40a4b2f547 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:26:00.786791 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687afb03-f219-4d3d-a46f-1c40a4b2f547 service nova] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:26:00.796601 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 710/1428] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:26:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:26:01.674332 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8c0cf-81f9-4bf8-8857-7ad8561cc596 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:01.674906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8c0cf-81f9-4bf8-8857-7ad8561cc596 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:01.674906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8c0cf-81f9-4bf8-8857-7ad8561cc596 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:01.676429 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 719/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 12:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:26:01.681501 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcfb44-1157-41fd-bef2-270449e6f84b None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:01.681934 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcfb44-1157-41fd-bef2-270449e6f84b None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:01.682063 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcfb44-1157-41fd-bef2-270449e6f84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:01.700409 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-d6bcfb44-1157-41fd-bef2-270449e6f84b None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:26:01.701649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-d6bcfb44-1157-41fd-bef2-270449e6f84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:26:01.709638 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 711/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 12:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:26:01.864261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8c0cf-81f9-4bf8-8857-7ad8561cc596 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:26:01.879047 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8c0cf-81f9-4bf8-8857-7ad8561cc596 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:26:01.884379 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738bc5e8-04c6-4b8d-a8d5-3c6cc0862c26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:26:01.885039 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738bc5e8-04c6-4b8d-a8d5-3c6cc0862c26 service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:01.885314 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738bc5e8-04c6-4b8d-a8d5-3c6cc0862c26 service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:01.885596 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738bc5e8-04c6-4b8d-a8d5-3c6cc0862c26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:01.894623 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738bc5e8-04c6-4b8d-a8d5-3c6cc0862c26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:26:01.896243 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738bc5e8-04c6-4b8d-a8d5-3c6cc0862c26 service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:26:01.905763 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 720/1431] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:26:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:26:02.772931 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dfd69a-1067-4b70-8d4f-ebed67274e61 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:02.773472 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dfd69a-1067-4b70-8d4f-ebed67274e61 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:02.773472 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dfd69a-1067-4b70-8d4f-ebed67274e61 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:02.774816 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 712/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 12:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:26:02.780252 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2712684-6f12-47e0-a4ea-dd19c8402087 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:02.780723 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2712684-6f12-47e0-a4ea-dd19c8402087 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:02.780835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2712684-6f12-47e0-a4ea-dd19c8402087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:02.801645 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-c2712684-6f12-47e0-a4ea-dd19c8402087 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:26:02.802871 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c2712684-6f12-47e0-a4ea-dd19c8402087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:26:02.810631 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 721/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 12:26:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:26:02.960709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dfd69a-1067-4b70-8d4f-ebed67274e61 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:26:02.975730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dfd69a-1067-4b70-8d4f-ebed67274e61 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:26:02.981212 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eed962-a214-4fce-95df-c63b83b63f57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:26:02.981883 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eed962-a214-4fce-95df-c63b83b63f57 service nova] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:02.982174 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eed962-a214-4fce-95df-c63b83b63f57 service nova] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:02.982456 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eed962-a214-4fce-95df-c63b83b63f57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:02.991067 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eed962-a214-4fce-95df-c63b83b63f57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:26:02.992733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eed962-a214-4fce-95df-c63b83b63f57 service nova] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:26:03.002597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 713/1434] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:26:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:26:03.874359 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea52819-9a51-48ab-ba85-80cdf7e6b3e2 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:03.874937 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea52819-9a51-48ab-ba85-80cdf7e6b3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:03.875038 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea52819-9a51-48ab-ba85-80cdf7e6b3e2 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:03.876432 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 722/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 12:26:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:26:03.881592 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31f5ad-d4a4-4acf-81d5-ad684a3351e7 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:03.882028 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31f5ad-d4a4-4acf-81d5-ad684a3351e7 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:03.882194 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31f5ad-d4a4-4acf-81d5-ad684a3351e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:03.900695 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-bf31f5ad-d4a4-4acf-81d5-ad684a3351e7 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:26:03.901891 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-bf31f5ad-d4a4-4acf-81d5-ad684a3351e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:26:03.909950 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 714/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 12:26:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:26:04.060509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea52819-9a51-48ab-ba85-80cdf7e6b3e2 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:26:04.076607 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea52819-9a51-48ab-ba85-80cdf7e6b3e2 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:26:04.083018 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7073090b-3288-4e11-b692-70961d21290b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:26:04.083668 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073090b-3288-4e11-b692-70961d21290b service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:04.083951 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073090b-3288-4e11-b692-70961d21290b service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:04.084265 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7073090b-3288-4e11-b692-70961d21290b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:04.092945 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7073090b-3288-4e11-b692-70961d21290b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:26:04.094637 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7073090b-3288-4e11-b692-70961d21290b service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:26:04.104512 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 723/1437] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:26:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:26:04.977686 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42e04d-7a97-40f3-8f9a-d7a2697201ea None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:04.978288 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42e04d-7a97-40f3-8f9a-d7a2697201ea None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:04.978288 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42e04d-7a97-40f3-8f9a-d7a2697201ea None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:04.979604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 715/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 12:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:26:04.985671 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19892a39-67c0-4ed7-915a-f632766cbcdd None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:04.986145 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19892a39-67c0-4ed7-915a-f632766cbcdd None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:04.986253 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19892a39-67c0-4ed7-915a-f632766cbcdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:05.004720 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-19892a39-67c0-4ed7-915a-f632766cbcdd None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:26:05.005922 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-19892a39-67c0-4ed7-915a-f632766cbcdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:26:05.013703 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 724/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 12:26:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:26:05.164493 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42e04d-7a97-40f3-8f9a-d7a2697201ea None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:26:05.179306 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42e04d-7a97-40f3-8f9a-d7a2697201ea None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:26:05.184943 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc786f6-708a-4c87-bd8c-100b7b1ebf84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:26:05.185582 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc786f6-708a-4c87-bd8c-100b7b1ebf84 service nova] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:05.185866 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc786f6-708a-4c87-bd8c-100b7b1ebf84 service nova] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:05.186703 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc786f6-708a-4c87-bd8c-100b7b1ebf84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:05.195005 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc786f6-708a-4c87-bd8c-100b7b1ebf84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:26:05.196887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc786f6-708a-4c87-bd8c-100b7b1ebf84 service nova] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:26:05.207103 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 716/1440] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:26:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:26:07.315180 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d01eb8-ac98-478c-912e-1389ba8243d9 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:07.315772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d01eb8-ac98-478c-912e-1389ba8243d9 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:07.315772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d01eb8-ac98-478c-912e-1389ba8243d9 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:07.317141 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 725/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 12:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:26:07.323131 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b52121-00d5-4380-830a-ed7365c37ba7 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:07.323621 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b52121-00d5-4380-830a-ed7365c37ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:07.323741 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b52121-00d5-4380-830a-ed7365c37ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:07.343651 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-05b52121-00d5-4380-830a-ed7365c37ba7 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:26:07.344957 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-05b52121-00d5-4380-830a-ed7365c37ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:26:07.352761 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 717/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 12:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:26:07.391348 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d01eb8-ac98-478c-912e-1389ba8243d9 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:26:07.393091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-81d01eb8-ac98-478c-912e-1389ba8243d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:26:07.406940 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d01eb8-ac98-478c-912e-1389ba8243d9 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:26:07.410557 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c279e90-54c1-4f7f-9c3a-1f2f57922d26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:26:07.411199 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c279e90-54c1-4f7f-9c3a-1f2f57922d26 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:07.411486 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c279e90-54c1-4f7f-9c3a-1f2f57922d26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:07.411770 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c279e90-54c1-4f7f-9c3a-1f2f57922d26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:07.421180 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c279e90-54c1-4f7f-9c3a-1f2f57922d26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:26:07.422605 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c279e90-54c1-4f7f-9c3a-1f2f57922d26 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:26:07.432555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 726/1443] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:26:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:26:09.908994 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ed614-c200-4eaa-990e-024c0a2d9297 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:09.909523 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ed614-c200-4eaa-990e-024c0a2d9297 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:09.909523 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ed614-c200-4eaa-990e-024c0a2d9297 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:09.910817 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 718/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 12:26:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:26:09.916029 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a7ba9-1552-4775-95b1-efc7d0672c7c None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:09.916479 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a7ba9-1552-4775-95b1-efc7d0672c7c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:09.916653 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a7ba9-1552-4775-95b1-efc7d0672c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:09.934004 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-601a7ba9-1552-4775-95b1-efc7d0672c7c None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:26:09.935190 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-601a7ba9-1552-4775-95b1-efc7d0672c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:26:09.943750 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 727/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 12:26:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:26:09.981378 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ed614-c200-4eaa-990e-024c0a2d9297 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:26:09.991981 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ed614-c200-4eaa-990e-024c0a2d9297 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:26:09.995527 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f72443-5675-4b19-8a42-93b309f4dfe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:26:09.996157 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f72443-5675-4b19-8a42-93b309f4dfe8 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:09.996473 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f72443-5675-4b19-8a42-93b309f4dfe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:09.996751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f72443-5675-4b19-8a42-93b309f4dfe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:10.006569 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f72443-5675-4b19-8a42-93b309f4dfe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:26:10.008002 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f72443-5675-4b19-8a42-93b309f4dfe8 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:26:10.018659 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 719/1446] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:26:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:26:12.550149 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70dc042-6edb-42be-a435-59635415bb96 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:12.550751 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70dc042-6edb-42be-a435-59635415bb96 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:12.550751 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70dc042-6edb-42be-a435-59635415bb96 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:12.552095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 728/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 12:26:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:26:12.557339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f8ad7-f567-4518-930a-1c17f4bb8170 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:12.557762 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f8ad7-f567-4518-930a-1c17f4bb8170 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:12.557920 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f8ad7-f567-4518-930a-1c17f4bb8170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:12.575825 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-2f2f8ad7-f567-4518-930a-1c17f4bb8170 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:26:12.577737 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-2f2f8ad7-f567-4518-930a-1c17f4bb8170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:26:12.585518 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 720/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 12:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:26:12.623657 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70dc042-6edb-42be-a435-59635415bb96 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:26:12.636132 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70dc042-6edb-42be-a435-59635415bb96 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:26:12.640785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5719074-83c3-4a9a-9c0e-19f3502be9f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:26:12.641444 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5719074-83c3-4a9a-9c0e-19f3502be9f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:12.641728 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5719074-83c3-4a9a-9c0e-19f3502be9f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:12.642008 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5719074-83c3-4a9a-9c0e-19f3502be9f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:12.651557 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5719074-83c3-4a9a-9c0e-19f3502be9f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:26:12.652998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5719074-83c3-4a9a-9c0e-19f3502be9f6 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:26:12.663288 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 729/1449] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:26:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:26:15.129449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6aa199-eebb-41f7-8c3f-9515bd752e99 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:15.130020 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6aa199-eebb-41f7-8c3f-9515bd752e99 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:15.130020 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6aa199-eebb-41f7-8c3f-9515bd752e99 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:15.131357 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 721/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 12:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:26:15.136598 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9acde33-5175-4ef5-bac7-1d7ca32a41c9 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:15.137053 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9acde33-5175-4ef5-bac7-1d7ca32a41c9 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:15.137293 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9acde33-5175-4ef5-bac7-1d7ca32a41c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:15.157058 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a9acde33-5175-4ef5-bac7-1d7ca32a41c9 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:26:15.158272 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a9acde33-5175-4ef5-bac7-1d7ca32a41c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:26:15.166247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 730/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 12:26:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:26:15.206443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6aa199-eebb-41f7-8c3f-9515bd752e99 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:26:15.217645 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6aa199-eebb-41f7-8c3f-9515bd752e99 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:26:15.221321 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82198a96-022f-4794-b839-3ae18ed5c863 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:26:15.221961 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82198a96-022f-4794-b839-3ae18ed5c863 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:15.222244 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82198a96-022f-4794-b839-3ae18ed5c863 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:15.222520 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82198a96-022f-4794-b839-3ae18ed5c863 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:15.231552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82198a96-022f-4794-b839-3ae18ed5c863 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:26:15.233421 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82198a96-022f-4794-b839-3ae18ed5c863 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:26:15.243124 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 722/1452] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:26:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:26:17.603213 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63702d17-4f68-4a93-8e07-f313eb42cbbd None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:17.603774 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63702d17-4f68-4a93-8e07-f313eb42cbbd None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:17.603774 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63702d17-4f68-4a93-8e07-f313eb42cbbd None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:17.605169 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 731/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 12:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:26:17.610606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e85906-bd53-44ce-9844-1619d5287525 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:17.611045 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e85906-bd53-44ce-9844-1619d5287525 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:17.611207 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e85906-bd53-44ce-9844-1619d5287525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:17.631630 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-05e85906-bd53-44ce-9844-1619d5287525 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:26:17.632864 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-05e85906-bd53-44ce-9844-1619d5287525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:26:17.641534 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 723/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 12:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:26:17.680020 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63702d17-4f68-4a93-8e07-f313eb42cbbd None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:26:17.691638 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63702d17-4f68-4a93-8e07-f313eb42cbbd None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:26:17.696032 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d692571-fce1-4b45-9864-a30d5349d296 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:26:17.696706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d692571-fce1-4b45-9864-a30d5349d296 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:17.696991 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d692571-fce1-4b45-9864-a30d5349d296 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:17.697268 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d692571-fce1-4b45-9864-a30d5349d296 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:17.707203 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d692571-fce1-4b45-9864-a30d5349d296 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:26:17.708681 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d692571-fce1-4b45-9864-a30d5349d296 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:26:17.718733 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 732/1455] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:26:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:26:20.016378 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8566a3-3050-4019-a70b-812b66137431 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:20.016963 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8566a3-3050-4019-a70b-812b66137431 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:20.016963 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8566a3-3050-4019-a70b-812b66137431 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:20.018421 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 724/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 12:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:26:20.023947 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9638166-abd7-4ba8-ba40-48c9be21f584 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:20.024423 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9638166-abd7-4ba8-ba40-48c9be21f584 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:20.024530 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9638166-abd7-4ba8-ba40-48c9be21f584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:20.042584 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-d9638166-abd7-4ba8-ba40-48c9be21f584 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:26:20.043780 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d9638166-abd7-4ba8-ba40-48c9be21f584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:26:20.052420 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 733/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 12:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:26:20.089490 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8566a3-3050-4019-a70b-812b66137431 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:26:20.095430 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=64 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:26:20.101901 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8566a3-3050-4019-a70b-812b66137431 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:26:20.105499 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755296db-b3eb-473a-ab9a-b680ab96d691 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:26:20.106139 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755296db-b3eb-473a-ab9a-b680ab96d691 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:20.106426 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755296db-b3eb-473a-ab9a-b680ab96d691 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:20.106708 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755296db-b3eb-473a-ab9a-b680ab96d691 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:20.116776 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755296db-b3eb-473a-ab9a-b680ab96d691 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:26:20.118254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755296db-b3eb-473a-ab9a-b680ab96d691 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:26:20.127808 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 725/1458] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:26:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:26:22.522439 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd134fd-77e9-4d96-8ce1-10e47a31ce41 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:22.522989 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd134fd-77e9-4d96-8ce1-10e47a31ce41 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:22.522989 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd134fd-77e9-4d96-8ce1-10e47a31ce41 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:22.524460 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 734/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 12:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:26:22.529928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc335c-1b05-4847-9e39-d02fbb96578a None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:22.530337 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc335c-1b05-4847-9e39-d02fbb96578a None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:22.530489 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc335c-1b05-4847-9e39-d02fbb96578a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:22.549232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-d5fc335c-1b05-4847-9e39-d02fbb96578a None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:26:22.550449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-d5fc335c-1b05-4847-9e39-d02fbb96578a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:26:22.558173 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 726/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 12:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:26:22.595708 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd134fd-77e9-4d96-8ce1-10e47a31ce41 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:26:22.601745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=73 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:26:22.608445 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd134fd-77e9-4d96-8ce1-10e47a31ce41 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:26:22.612218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d9210-c55e-44bf-aab9-8f4f36b9ad96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:26:22.612888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d9210-c55e-44bf-aab9-8f4f36b9ad96 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:22.613170 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d9210-c55e-44bf-aab9-8f4f36b9ad96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:22.613450 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d9210-c55e-44bf-aab9-8f4f36b9ad96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:22.622461 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d9210-c55e-44bf-aab9-8f4f36b9ad96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:26:22.623887 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487d9210-c55e-44bf-aab9-8f4f36b9ad96 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:26:22.633632 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 735/1461] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:26:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:26:24.994741 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c5d6e-0958-496c-b022-0df439b231d1 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:24.995359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c5d6e-0958-496c-b022-0df439b231d1 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:24.995359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c5d6e-0958-496c-b022-0df439b231d1 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:24.996788 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 727/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 12:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:26:25.004702 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe71bca-b276-423d-8cf9-832ecccc59d0 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:25.005205 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe71bca-b276-423d-8cf9-832ecccc59d0 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:25.005375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe71bca-b276-423d-8cf9-832ecccc59d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:25.025598 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-6fe71bca-b276-423d-8cf9-832ecccc59d0 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:26:25.027302 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-6fe71bca-b276-423d-8cf9-832ecccc59d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:26:25.038604 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 736/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 12:26:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:26:25.089823 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c5d6e-0958-496c-b022-0df439b231d1 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:26:25.102994 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c5d6e-0958-496c-b022-0df439b231d1 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:26:25.114035 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d8c04-47c3-4faa-8267-f874c1bac9d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:26:25.115138 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d8c04-47c3-4faa-8267-f874c1bac9d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:25.115617 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d8c04-47c3-4faa-8267-f874c1bac9d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:25.116115 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d8c04-47c3-4faa-8267-f874c1bac9d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:25.130697 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d8c04-47c3-4faa-8267-f874c1bac9d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:26:25.133127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d8c04-47c3-4faa-8267-f874c1bac9d8 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:26:25.148704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 728/1464] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:26:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:26:27.431091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a87b41-83f8-47db-8a43-d15124a8bcf4 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:27.432046 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a87b41-83f8-47db-8a43-d15124a8bcf4 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:27.432046 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a87b41-83f8-47db-8a43-d15124a8bcf4 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:27.433255 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 737/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 12:26:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:26:27.439978 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2880ad7-b4f5-4e43-8753-7ff88f0a4212 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:27.440656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2880ad7-b4f5-4e43-8753-7ff88f0a4212 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:27.440656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2880ad7-b4f5-4e43-8753-7ff88f0a4212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:27.464657 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-f2880ad7-b4f5-4e43-8753-7ff88f0a4212 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:26:27.466481 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-f2880ad7-b4f5-4e43-8753-7ff88f0a4212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:26:27.475510 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 729/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 12:26:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:26:27.512712 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a87b41-83f8-47db-8a43-d15124a8bcf4 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:26:27.523531 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a87b41-83f8-47db-8a43-d15124a8bcf4 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:26:27.527042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a83bbd-8bf6-4a40-9f6e-3079bbc69276 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:26:27.527680 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a83bbd-8bf6-4a40-9f6e-3079bbc69276 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:27.527960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a83bbd-8bf6-4a40-9f6e-3079bbc69276 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:27.528265 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a83bbd-8bf6-4a40-9f6e-3079bbc69276 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:27.537869 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a83bbd-8bf6-4a40-9f6e-3079bbc69276 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:26:27.539282 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a83bbd-8bf6-4a40-9f6e-3079bbc69276 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:26:27.549550 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 738/1467] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:26:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:26:29.843444 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f116fa5-f504-49d6-a59c-9456599577f5 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:29.844042 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f116fa5-f504-49d6-a59c-9456599577f5 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:29.844042 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f116fa5-f504-49d6-a59c-9456599577f5 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:29.845653 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 730/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 12:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:26:29.852199 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16079eb0-0944-475b-b2a3-7e0931fbe5fd None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:29.852721 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16079eb0-0944-475b-b2a3-7e0931fbe5fd None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:29.852989 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16079eb0-0944-475b-b2a3-7e0931fbe5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:29.871256 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-16079eb0-0944-475b-b2a3-7e0931fbe5fd None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:26:29.872489 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-16079eb0-0944-475b-b2a3-7e0931fbe5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:26:29.881004 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 739/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 12:26:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:26:29.918192 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f116fa5-f504-49d6-a59c-9456599577f5 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:26:29.930924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f116fa5-f504-49d6-a59c-9456599577f5 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:26:29.934560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0314ec17-986f-44c7-b6b6-cc559e08016c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:26:29.935238 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0314ec17-986f-44c7-b6b6-cc559e08016c service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:26:29.935530 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0314ec17-986f-44c7-b6b6-cc559e08016c service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:26:29.935809 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0314ec17-986f-44c7-b6b6-cc559e08016c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:26:29.946422 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0314ec17-986f-44c7-b6b6-cc559e08016c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:26:29.947867 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0314ec17-986f-44c7-b6b6-cc559e08016c service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:26:29.958628 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 731/1470] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:26:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:26:39.865886 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=17 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:26:39.953354 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=23 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:27:04.354315 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4580a-a58f-4cc9-9d13-b12fc35e02c8 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:27:04.354951 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4580a-a58f-4cc9-9d13-b12fc35e02c8 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:27:04.354951 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4580a-a58f-4cc9-9d13-b12fc35e02c8 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:27:04.356562 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 740/1471] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:27:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:27:04.362161 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ceaa7f-931e-40da-8db2-02b8663f01bd None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:27:04.362684 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ceaa7f-931e-40da-8db2-02b8663f01bd None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:27:04.362794 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ceaa7f-931e-40da-8db2-02b8663f01bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:27:04.381514 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-64ceaa7f-931e-40da-8db2-02b8663f01bd None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:27:04.382806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-64ceaa7f-931e-40da-8db2-02b8663f01bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:27:04.391978 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 732/1472] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:27:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:27:04.443397 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7c0a3-a4ac-4cf8-941d-b9b6c472edd0 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:27:04.443713 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7c0a3-a4ac-4cf8-941d-b9b6c472edd0 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:27:04.443992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba7c0a3-a4ac-4cf8-941d-b9b6c472edd0 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:27:04.445391 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 741/1473] 10.222.0.16 () {54 vars in 962 bytes} [Mon May 23 12:27:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:27:04.450473 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799e6d7-f253-4d8a-8e88-d01fed92cece None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:27:04.450764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799e6d7-f253-4d8a-8e88-d01fed92cece None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:27:04.451033 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799e6d7-f253-4d8a-8e88-d01fed92cece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:27:04.470433 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-7799e6d7-f253-4d8a-8e88-d01fed92cece None None] MFA Rules not processed for user `2483a351f5e943ad8c872a98a4717b0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:27:04.471609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-7799e6d7-f253-4d8a-8e88-d01fed92cece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:27:04.479104 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 733/1474] 10.222.0.16 () {58 vars in 1059 bytes} [Mon May 23 12:27:04 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:27:04.486989 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7c0a3-a4ac-4cf8-941d-b9b6c472edd0 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:27:04.497778 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba7c0a3-a4ac-4cf8-941d-b9b6c472edd0 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:27:04.502020 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da37ff0-6de3-4b04-925b-b54143c615c3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2483a351f5e943ad8c872a98a4717b0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:27:04.502659 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da37ff0-6de3-4b04-925b-b54143c615c3 service glance] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:27:04.502944 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da37ff0-6de3-4b04-925b-b54143c615c3 service glance] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:27:04.503223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da37ff0-6de3-4b04-925b-b54143c615c3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:27:04.512187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da37ff0-6de3-4b04-925b-b54143c615c3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:27:04.513620 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da37ff0-6de3-4b04-925b-b54143c615c3 service glance] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:27:04.523722 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 742/1475] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:27:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:27:06.863375 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da773a9-a21c-4e51-bf3e-8bbd0e78e5ba None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:27:06.864098 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da773a9-a21c-4e51-bf3e-8bbd0e78e5ba None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:27:06.864098 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da773a9-a21c-4e51-bf3e-8bbd0e78e5ba None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:27:06.865338 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 734/1476] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:27:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:27:06.870692 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172df4c-ca7a-461d-be1a-8ce9e320f5ac None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:27:06.871167 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172df4c-ca7a-461d-be1a-8ce9e320f5ac None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:27:06.871264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172df4c-ca7a-461d-be1a-8ce9e320f5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:27:06.889212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-3172df4c-ca7a-461d-be1a-8ce9e320f5ac None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:27:06.890425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3172df4c-ca7a-461d-be1a-8ce9e320f5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:27:06.899642 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 743/1477] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:27:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:27:06.923207 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da773a9-a21c-4e51-bf3e-8bbd0e78e5ba None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:27:06.936333 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da773a9-a21c-4e51-bf3e-8bbd0e78e5ba None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:27:06.942024 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1eb55-7fcf-49e7-bc08-554fab432c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:27:06.943119 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1eb55-7fcf-49e7-bc08-554fab432c7a admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:27:06.943600 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1eb55-7fcf-49e7-bc08-554fab432c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:27:06.944087 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1eb55-7fcf-49e7-bc08-554fab432c7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:27:06.945280 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1eb55-7fcf-49e7-bc08-554fab432c7a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:27:06.947537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: WARNING py.warnings [None req-c1f1eb55-7fcf-49e7-bc08-554fab432c7a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 12:27:06.947537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: warnings.warn(msg) May 23 12:27:06.947537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: May 23 12:27:06.948721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1eb55-7fcf-49e7-bc08-554fab432c7a admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:27:06.969054 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 735/1478] 10.222.0.16 () {56 vars in 1108 bytes} [Mon May 23 12:27:06 2022] GET /identity/v3/projects => generated 2553 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:27:09.211206 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40941a2-91bc-4377-ba9c-ae77a2ad4806 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:27:09.211850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40941a2-91bc-4377-ba9c-ae77a2ad4806 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:27:09.211850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40941a2-91bc-4377-ba9c-ae77a2ad4806 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:27:09.213155 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 744/1479] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:27:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:27:09.218915 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fb2e8-1ad7-4596-b523-676d153c2ba1 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:27:09.219331 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fb2e8-1ad7-4596-b523-676d153c2ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:27:09.219617 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fb2e8-1ad7-4596-b523-676d153c2ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:27:09.246012 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-1b6fb2e8-1ad7-4596-b523-676d153c2ba1 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:27:09.247849 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-1b6fb2e8-1ad7-4596-b523-676d153c2ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:27:09.256649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 736/1480] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:27:09.304075 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40941a2-91bc-4377-ba9c-ae77a2ad4806 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:27:09.318298 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40941a2-91bc-4377-ba9c-ae77a2ad4806 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:27:09.323923 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85507041-6be9-49ba-95e3-0d385815c678 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:27:09.325195 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85507041-6be9-49ba-95e3-0d385815c678 service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:27:09.325195 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85507041-6be9-49ba-95e3-0d385815c678 service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:27:09.325195 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85507041-6be9-49ba-95e3-0d385815c678 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:27:09.334281 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85507041-6be9-49ba-95e3-0d385815c678 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:27:09.335862 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85507041-6be9-49ba-95e3-0d385815c678 service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:27:09.346047 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 745/1481] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:27:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:27:11.806898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e10466-20ef-476b-b1cd-4f2992a12630 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:27:11.807480 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e10466-20ef-476b-b1cd-4f2992a12630 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:27:11.807480 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e10466-20ef-476b-b1cd-4f2992a12630 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:27:11.809374 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 737/1482] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:27:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:27:11.815298 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e096b-7982-415e-b389-a4da0641a715 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:27:11.815746 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e096b-7982-415e-b389-a4da0641a715 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:27:11.815882 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e096b-7982-415e-b389-a4da0641a715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:27:11.833844 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-037e096b-7982-415e-b389-a4da0641a715 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:27:11.835043 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-037e096b-7982-415e-b389-a4da0641a715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:27:11.842947 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 746/1483] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:27:11.880200 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e10466-20ef-476b-b1cd-4f2992a12630 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:27:11.891654 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e10466-20ef-476b-b1cd-4f2992a12630 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:27:11.895429 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47617b-1532-42ec-ba64-edec0194650b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:27:11.896071 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47617b-1532-42ec-ba64-edec0194650b service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:27:11.896383 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47617b-1532-42ec-ba64-edec0194650b service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:27:11.896660 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47617b-1532-42ec-ba64-edec0194650b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:27:11.905714 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a47617b-1532-42ec-ba64-edec0194650b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:27:11.907164 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a47617b-1532-42ec-ba64-edec0194650b service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:27:11.917474 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 738/1484] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:27:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:27:14.265437 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd4a7a-ee03-4558-89a8-4e1ec5402bfa None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:27:14.266054 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd4a7a-ee03-4558-89a8-4e1ec5402bfa None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:27:14.266054 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd4a7a-ee03-4558-89a8-4e1ec5402bfa None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:27:14.268022 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 747/1485] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:27:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:27:14.274024 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef8ab6-bbf2-46ad-90b7-f87485308ada None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:27:14.274530 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef8ab6-bbf2-46ad-90b7-f87485308ada None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:27:14.274631 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef8ab6-bbf2-46ad-90b7-f87485308ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:27:14.293566 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-69ef8ab6-bbf2-46ad-90b7-f87485308ada None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:27:14.294640 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-69ef8ab6-bbf2-46ad-90b7-f87485308ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:27:14.303353 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 739/1486] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:27:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:27:14.361777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd4a7a-ee03-4558-89a8-4e1ec5402bfa None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:27:14.378843 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd4a7a-ee03-4558-89a8-4e1ec5402bfa None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:27:14.384603 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf9b2d-f6a7-4b6d-bd2b-677e4e8c5094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:27:14.385541 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf9b2d-f6a7-4b6d-bd2b-677e4e8c5094 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:27:14.385831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf9b2d-f6a7-4b6d-bd2b-677e4e8c5094 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:27:14.386252 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf9b2d-f6a7-4b6d-bd2b-677e4e8c5094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:27:14.397871 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf9b2d-f6a7-4b6d-bd2b-677e4e8c5094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:27:14.400964 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf9b2d-f6a7-4b6d-bd2b-677e4e8c5094 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:27:14.416258 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 748/1487] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:27:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:27:16.942387 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446caa1c-4028-4ca7-97e4-2a71e34d9754 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:27:16.942971 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446caa1c-4028-4ca7-97e4-2a71e34d9754 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:27:16.942971 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446caa1c-4028-4ca7-97e4-2a71e34d9754 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:27:16.944419 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 740/1488] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:27:16.950039 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466940c7-1daf-478a-bdb7-e8dc2d54799f None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:27:16.950503 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466940c7-1daf-478a-bdb7-e8dc2d54799f None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:27:16.950615 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466940c7-1daf-478a-bdb7-e8dc2d54799f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:27:16.968890 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-466940c7-1daf-478a-bdb7-e8dc2d54799f None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:27:16.970613 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-466940c7-1daf-478a-bdb7-e8dc2d54799f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:27:16.978522 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 749/1489] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:27:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:27:17.019225 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446caa1c-4028-4ca7-97e4-2a71e34d9754 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:27:17.030897 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446caa1c-4028-4ca7-97e4-2a71e34d9754 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:27:17.035486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273f47a-abb9-4b70-8320-514deb911ca1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:27:17.036126 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273f47a-abb9-4b70-8320-514deb911ca1 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:27:17.036448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273f47a-abb9-4b70-8320-514deb911ca1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:27:17.036732 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273f47a-abb9-4b70-8320-514deb911ca1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:27:17.046123 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1273f47a-abb9-4b70-8320-514deb911ca1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:27:17.047566 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1273f47a-abb9-4b70-8320-514deb911ca1 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:27:17.058139 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 741/1490] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:27:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:27:19.615410 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec51001-b1db-4e7f-9552-b17f49e7c9b0 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:27:19.615978 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec51001-b1db-4e7f-9552-b17f49e7c9b0 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:27:19.615978 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec51001-b1db-4e7f-9552-b17f49e7c9b0 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:27:19.617411 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 750/1491] 10.222.0.16 () {54 vars in 926 bytes} [Mon May 23 12:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:27:19.622802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba04c8-4f41-403d-a814-0d02590ea0ff None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:27:19.623307 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba04c8-4f41-403d-a814-0d02590ea0ff None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:27:19.623425 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba04c8-4f41-403d-a814-0d02590ea0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:27:19.641755 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-3dba04c8-4f41-403d-a814-0d02590ea0ff None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:27:19.642972 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-3dba04c8-4f41-403d-a814-0d02590ea0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:27:19.650858 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 742/1492] 10.222.0.16 () {58 vars in 1023 bytes} [Mon May 23 12:27:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:27:19.690365 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec51001-b1db-4e7f-9552-b17f49e7c9b0 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:27:19.701495 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec51001-b1db-4e7f-9552-b17f49e7c9b0 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:27:19.705351 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f1dc99-27ae-4e85-b052-8ae43247d367 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:27:19.706000 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1dc99-27ae-4e85-b052-8ae43247d367 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:27:19.706285 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1dc99-27ae-4e85-b052-8ae43247d367 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:27:19.706565 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f1dc99-27ae-4e85-b052-8ae43247d367 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:27:19.716410 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f1dc99-27ae-4e85-b052-8ae43247d367 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:27:19.717850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f1dc99-27ae-4e85-b052-8ae43247d367 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:27:19.728678 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 751/1493] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:27:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:27:29.636682 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=33 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:27:29.724387 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=33 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:29:08.876968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687e7e2-d5af-427a-8440-99d08eedbd08 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:29:08.877538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687e7e2-d5af-427a-8440-99d08eedbd08 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:29:08.877538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687e7e2-d5af-427a-8440-99d08eedbd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:29:08.895609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-c687e7e2-d5af-427a-8440-99d08eedbd08 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:29:08.896858 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-c687e7e2-d5af-427a-8440-99d08eedbd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:29:08.905566 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 743/1494] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:29:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:29:08.915016 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847cfdd-91a3-4935-918e-935ba9cfa8de None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:29:08.915588 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847cfdd-91a3-4935-918e-935ba9cfa8de None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:29:08.915588 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847cfdd-91a3-4935-918e-935ba9cfa8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:29:08.934249 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-e847cfdd-91a3-4935-918e-935ba9cfa8de None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:29:08.935499 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e847cfdd-91a3-4935-918e-935ba9cfa8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:29:08.945454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 752/1495] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:29:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:29:08.950325 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687e7e2-d5af-427a-8440-99d08eedbd08 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:29:08.960009 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687e7e2-d5af-427a-8440-99d08eedbd08 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:29:08.964711 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18f265-3d74-4889-ab78-d158b1bbbfcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:29:08.965369 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18f265-3d74-4889-ab78-d158b1bbbfcb admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:29:08.965659 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18f265-3d74-4889-ab78-d158b1bbbfcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:29:08.965943 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18f265-3d74-4889-ab78-d158b1bbbfcb admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:29:08.967204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d18f265-3d74-4889-ab78-d158b1bbbfcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:29:08.967518 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d18f265-3d74-4889-ab78-d158b1bbbfcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:29:08.968579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: WARNING py.warnings [None req-5d18f265-3d74-4889-ab78-d158b1bbbfcb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 12:29:08.968579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: warnings.warn(msg) May 23 12:29:08.968579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: May 23 12:29:08.969469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d18f265-3d74-4889-ab78-d158b1bbbfcb admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:29:08.981163 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 744/1496] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:29:08.990571 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847cfdd-91a3-4935-918e-935ba9cfa8de None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:29:09.006614 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847cfdd-91a3-4935-918e-935ba9cfa8de None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:29:09.020072 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2828b2-bf17-41b5-956b-ba339c88024d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:29:09.021219 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2828b2-bf17-41b5-956b-ba339c88024d admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:29:09.021564 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2828b2-bf17-41b5-956b-ba339c88024d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:29:09.021903 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2828b2-bf17-41b5-956b-ba339c88024d admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:29:09.026104 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2828b2-bf17-41b5-956b-ba339c88024d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:29:09.028001 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: WARNING py.warnings [None req-9e2828b2-bf17-41b5-956b-ba339c88024d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 12:29:09.028001 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: warnings.warn(msg) May 23 12:29:09.028001 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: May 23 12:29:09.029800 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2828b2-bf17-41b5-956b-ba339c88024d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:29:09.072189 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 753/1497] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:29:08 2022] POST /identity/v3/projects => generated 377 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:29:09.077555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18f265-3d74-4889-ab78-d158b1bbbfcb admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:29:09.087654 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18f265-3d74-4889-ab78-d158b1bbbfcb admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:29:09.144448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d04ab-4d09-4441-abbf-5e7f49215f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:29:09.145046 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d04ab-4d09-4441-abbf-5e7f49215f8c admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:29:09.145333 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d04ab-4d09-4441-abbf-5e7f49215f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:29:09.145614 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d04ab-4d09-4441-abbf-5e7f49215f8c admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:29:09.146304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d04ab-4d09-4441-abbf-5e7f49215f8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:29:09.147344 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: WARNING py.warnings [None req-c57d04ab-4d09-4441-abbf-5e7f49215f8c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 12:29:09.147344 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: warnings.warn(msg) May 23 12:29:09.147344 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: May 23 12:29:09.148587 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d04ab-4d09-4441-abbf-5e7f49215f8c admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:29:09.168502 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 745/1498] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:29:09 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:29:09.172832 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2828b2-bf17-41b5-956b-ba339c88024d admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:29:09.183083 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2828b2-bf17-41b5-956b-ba339c88024d admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:29:09.187816 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c41d16-a530-4533-9a67-8a69757fe4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:29:09.188520 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c41d16-a530-4533-9a67-8a69757fe4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:29:09.188804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c41d16-a530-4533-9a67-8a69757fe4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:29:09.189082 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c41d16-a530-4533-9a67-8a69757fe4d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:29:09.189798 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c41d16-a530-4533-9a67-8a69757fe4d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:29:09.190804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: WARNING py.warnings [None req-75c41d16-a530-4533-9a67-8a69757fe4d3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 12:29:09.190804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: warnings.warn(msg) May 23 12:29:09.190804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: May 23 12:29:09.191676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c41d16-a530-4533-9a67-8a69757fe4d3 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:29:09.205375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 754/1499] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:29:09.209717 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d04ab-4d09-4441-abbf-5e7f49215f8c admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:29:09.220778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d04ab-4d09-4441-abbf-5e7f49215f8c admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:29:09.225360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343dcc39-e1bc-45ab-b38f-1263fac72f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:29:09.226020 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343dcc39-e1bc-45ab-b38f-1263fac72f22 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:29:09.226432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343dcc39-e1bc-45ab-b38f-1263fac72f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:29:09.226711 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343dcc39-e1bc-45ab-b38f-1263fac72f22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:29:09.227437 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343dcc39-e1bc-45ab-b38f-1263fac72f22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:29:09.228494 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: WARNING py.warnings [None req-343dcc39-e1bc-45ab-b38f-1263fac72f22 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 12:29:09.228494 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: warnings.warn(msg) May 23 12:29:09.228494 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: May 23 12:29:09.229382 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343dcc39-e1bc-45ab-b38f-1263fac72f22 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:29:09.243238 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 746/1500] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:29:09.247470 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c41d16-a530-4533-9a67-8a69757fe4d3 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:29:09.258135 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c41d16-a530-4533-9a67-8a69757fe4d3 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:29:09.262699 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f97af-ef5b-4859-b659-f8771fd52aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:29:09.264076 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f97af-ef5b-4859-b659-f8771fd52aa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:29:09.264784 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f97af-ef5b-4859-b659-f8771fd52aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:29:09.265003 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f97af-ef5b-4859-b659-f8771fd52aa2 admin admin] PATH_INFO: `/v3/projects/303a23b23db646a3812632a332a55ea6/users/44db1835617342ad96cee84f8e8bafd2/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:29:09.281084 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05f97af-ef5b-4859-b659-f8771fd52aa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=303a23b23db646a3812632a332a55ea6, user_id=44db1835617342ad96cee84f8e8bafd2, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:29:09.282121 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: WARNING py.warnings [None req-a05f97af-ef5b-4859-b659-f8771fd52aa2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 12:29:09.282121 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: warnings.warn(msg) May 23 12:29:09.282121 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: May 23 12:29:09.284038 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05f97af-ef5b-4859-b659-f8771fd52aa2 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:29:09.297819 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 755/1501] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:29:09 2022] PUT /identity/v3/projects/303a23b23db646a3812632a332a55ea6/users/44db1835617342ad96cee84f8e8bafd2/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:29:09.309065 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fc5a3-ed9e-4b4a-90f5-433a3026cfa2 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:29:09.309362 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fc5a3-ed9e-4b4a-90f5-433a3026cfa2 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:29:09.309644 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fc5a3-ed9e-4b4a-90f5-433a3026cfa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:29:09.338286 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-cf3fc5a3-ed9e-4b4a-90f5-433a3026cfa2 None None] MFA Rules not processed for user `44db1835617342ad96cee84f8e8bafd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:29:09.339509 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-cf3fc5a3-ed9e-4b4a-90f5-433a3026cfa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:29:09.405608 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 747/1502] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:29:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:29:09.418766 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd249c2d-7809-48b7-8b42-29ca2e639a39 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:29:09.419062 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd249c2d-7809-48b7-8b42-29ca2e639a39 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:29:09.419346 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd249c2d-7809-48b7-8b42-29ca2e639a39 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:29:09.420917 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 756/1503] 10.222.0.16 () {54 vars in 963 bytes} [Mon May 23 12:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:29:09.426394 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0784cf-f5fb-44d5-9a66-ee268624a732 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:29:09.426791 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0784cf-f5fb-44d5-9a66-ee268624a732 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:29:09.427086 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0784cf-f5fb-44d5-9a66-ee268624a732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:29:09.454678 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-ba0784cf-f5fb-44d5-9a66-ee268624a732 None None] MFA Rules not processed for user `e12ef3e115544a51a6bf9b08d265fefb`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:29:09.455870 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-ba0784cf-f5fb-44d5-9a66-ee268624a732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:29:09.502105 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:29:09.512442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 748/1504] 10.222.0.16 () {58 vars in 1060 bytes} [Mon May 23 12:29:09 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:29:09.520907 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd249c2d-7809-48b7-8b42-29ca2e639a39 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:29:09.531802 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd249c2d-7809-48b7-8b42-29ca2e639a39 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:29:09.535353 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a3f5d-ad99-47bd-a4d6-5db70e7f9125 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:29:09.535991 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a3f5d-ad99-47bd-a4d6-5db70e7f9125 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:29:09.536303 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a3f5d-ad99-47bd-a4d6-5db70e7f9125 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:29:09.536584 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a3f5d-ad99-47bd-a4d6-5db70e7f9125 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:29:09.545083 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a3f5d-ad99-47bd-a4d6-5db70e7f9125 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:29:09.546506 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a3f5d-ad99-47bd-a4d6-5db70e7f9125 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:29:09.555382 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 757/1505] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:29:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:29:09.599650 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7af250-b227-438c-bf29-acac197d5383 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:29:09.599938 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7af250-b227-438c-bf29-acac197d5383 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:29:09.600240 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7af250-b227-438c-bf29-acac197d5383 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:29:09.601536 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 749/1506] 10.222.0.16 () {52 vars in 829 bytes} [Mon May 23 12:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:29:09.605156 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a3f5d-ad99-47bd-a4d6-5db70e7f9125 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:29:09.669617 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a3f5d-ad99-47bd-a4d6-5db70e7f9125 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:29:09.674393 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df69cf-1038-4f19-8d25-3949021bc757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:29:09.675259 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df69cf-1038-4f19-8d25-3949021bc757 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:29:09.675539 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df69cf-1038-4f19-8d25-3949021bc757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:29:09.675816 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df69cf-1038-4f19-8d25-3949021bc757 admin admin] PATH_INFO: `/v3/users/44db1835617342ad96cee84f8e8bafd2` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:29:09.677033 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df69cf-1038-4f19-8d25-3949021bc757 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44db1835617342ad96cee84f8e8bafd2)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:29:09.678034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: WARNING py.warnings [None req-c4df69cf-1038-4f19-8d25-3949021bc757 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 12:29:09.678034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: warnings.warn(msg) May 23 12:29:09.678034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: May 23 12:29:09.679247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df69cf-1038-4f19-8d25-3949021bc757 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:29:09.747547 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c4df69cf-1038-4f19-8d25-3949021bc757 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44db1835617342ad96cee84f8e8bafd2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:29:09.767884 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c4df69cf-1038-4f19-8d25-3949021bc757 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44db1835617342ad96cee84f8e8bafd2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:29:09.777168 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c4df69cf-1038-4f19-8d25-3949021bc757 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44db1835617342ad96cee84f8e8bafd2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:29:09.777802 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c4df69cf-1038-4f19-8d25-3949021bc757 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44db1835617342ad96cee84f8e8bafd2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:29:09.778496 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c4df69cf-1038-4f19-8d25-3949021bc757 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44db1835617342ad96cee84f8e8bafd2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:29:09.787447 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:29:09.799281 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 758/1507] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:29:09 2022] DELETE /identity/v3/users/44db1835617342ad96cee84f8e8bafd2 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:29:09.803773 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fddce47-3854-4816-a7f0-8a5ef89e1349 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:29:09.804069 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fddce47-3854-4816-a7f0-8a5ef89e1349 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:29:09.804417 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fddce47-3854-4816-a7f0-8a5ef89e1349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:29:09.823072 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-3fddce47-3854-4816-a7f0-8a5ef89e1349 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:29:09.824617 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-3fddce47-3854-4816-a7f0-8a5ef89e1349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:29:09.886517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 750/1508] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:29:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:29:09.896294 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df69cf-1038-4f19-8d25-3949021bc757 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:29:09.897744 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c4df69cf-1038-4f19-8d25-3949021bc757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:29:09.958325 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df69cf-1038-4f19-8d25-3949021bc757 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:29:09.961793 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81e33f-f7ca-4449-940e-79ab9fd5845a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:29:09.962441 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81e33f-f7ca-4449-940e-79ab9fd5845a service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:29:09.962726 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81e33f-f7ca-4449-940e-79ab9fd5845a service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:29:09.963001 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81e33f-f7ca-4449-940e-79ab9fd5845a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:29:09.972648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81e33f-f7ca-4449-940e-79ab9fd5845a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:29:09.974096 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81e33f-f7ca-4449-940e-79ab9fd5845a service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:29:09.983418 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 759/1509] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:29:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:29:10.332612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fddce47-3854-4816-a7f0-8a5ef89e1349 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:29:10.334214 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-3fddce47-3854-4816-a7f0-8a5ef89e1349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:29:10.350006 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fddce47-3854-4816-a7f0-8a5ef89e1349 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:29:10.354765 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034dd4b-f140-4a3f-92a7-92682d713839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:29:10.355666 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034dd4b-f140-4a3f-92a7-92682d713839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:29:10.355959 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034dd4b-f140-4a3f-92a7-92682d713839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:29:10.356289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034dd4b-f140-4a3f-92a7-92682d713839 admin admin] PATH_INFO: `/v3/projects/303a23b23db646a3812632a332a55ea6` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:29:10.357578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2034dd4b-f140-4a3f-92a7-92682d713839 admin admin] RBAC: Authorizing `identity:delete_project(project_id=303a23b23db646a3812632a332a55ea6)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:29:10.359434 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: WARNING py.warnings [None req-2034dd4b-f140-4a3f-92a7-92682d713839 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 12:29:10.359434 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: warnings.warn(msg) May 23 12:29:10.359434 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: May 23 12:29:10.360764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2034dd4b-f140-4a3f-92a7-92682d713839 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:29:10.425291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2034dd4b-f140-4a3f-92a7-92682d713839 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '303a23b23db646a3812632a332a55ea6', 'request_id': 'req-2034dd4b-f140-4a3f-92a7-92682d713839'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:29:10.434638 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2034dd4b-f140-4a3f-92a7-92682d713839 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '303a23b23db646a3812632a332a55ea6', 'request_id': 'req-2034dd4b-f140-4a3f-92a7-92682d713839'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:29:10.454799 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2034dd4b-f140-4a3f-92a7-92682d713839 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '303a23b23db646a3812632a332a55ea6', 'request_id': 'req-2034dd4b-f140-4a3f-92a7-92682d713839'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:29:10.466878 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2034dd4b-f140-4a3f-92a7-92682d713839 admin admin] The token cache is being invalidate because project 303a23b23db646a3812632a332a55ea6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:29:10.467270 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2034dd4b-f140-4a3f-92a7-92682d713839 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:29:10.468971 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2034dd4b-f140-4a3f-92a7-92682d713839 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:29:10.470727 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 751/1510] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:29:10 2022] DELETE /identity/v3/projects/303a23b23db646a3812632a332a55ea6 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:29:19.979288 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=29 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:29:20.464206 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:30:58.688403 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb59a5-a3c2-45a5-8410-132d9f10830f None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:30:58.688981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb59a5-a3c2-45a5-8410-132d9f10830f None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:30:58.688981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb59a5-a3c2-45a5-8410-132d9f10830f None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:30:58.690420 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 760/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 12:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:30:58.695868 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121925e6-a619-4c09-a46e-5053d95f7912 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:30:58.696396 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121925e6-a619-4c09-a46e-5053d95f7912 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:30:58.696503 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121925e6-a619-4c09-a46e-5053d95f7912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:30:58.716731 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-121925e6-a619-4c09-a46e-5053d95f7912 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:30:58.717971 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-121925e6-a619-4c09-a46e-5053d95f7912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:30:58.786638 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 752/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 12:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:30:58.960531 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb59a5-a3c2-45a5-8410-132d9f10830f None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:30:58.962213 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b9eb59a5-a3c2-45a5-8410-132d9f10830f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:30:59.053633 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb59a5-a3c2-45a5-8410-132d9f10830f None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:30:59.060340 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eea023-7399-4f9b-aa83-3dc616ffe310 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:30:59.061060 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eea023-7399-4f9b-aa83-3dc616ffe310 service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:30:59.061359 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eea023-7399-4f9b-aa83-3dc616ffe310 service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:30:59.061691 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eea023-7399-4f9b-aa83-3dc616ffe310 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:30:59.070861 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eea023-7399-4f9b-aa83-3dc616ffe310 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:30:59.072534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eea023-7399-4f9b-aa83-3dc616ffe310 service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:30:59.084278 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 761/1513] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:30:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:30:59.354234 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0df7b-56b4-4188-b7a6-5748859c086d None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:30:59.354579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0df7b-56b4-4188-b7a6-5748859c086d None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:30:59.354817 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0df7b-56b4-4188-b7a6-5748859c086d None None] PATH_INFO: `/v3` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:30:59.356896 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 753/1514] 10.222.0.15 () {54 vars in 925 bytes} [Mon May 23 12:30:59 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:30:59.363724 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189d3a5-8dcb-419f-9cb8-9548b4cdff4d None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:30:59.364029 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189d3a5-8dcb-419f-9cb8-9548b4cdff4d None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:30:59.364345 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189d3a5-8dcb-419f-9cb8-9548b4cdff4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:30:59.389009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-2189d3a5-8dcb-419f-9cb8-9548b4cdff4d None None] MFA Rules not processed for user `ebc0f137e0d24dd2a460689bc17a1b98`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:30:59.390291 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-2189d3a5-8dcb-419f-9cb8-9548b4cdff4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:30:59.462963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 762/1515] 10.222.0.15 () {58 vars in 1015 bytes} [Mon May 23 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:31:00.000903 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6aee21-469e-4763-b8b5-9412f553f5e6 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:31:00.001470 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6aee21-469e-4763-b8b5-9412f553f5e6 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:31:00.001470 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6aee21-469e-4763-b8b5-9412f553f5e6 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:31:00.002884 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 754/1516] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 12:30:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:31:00.007970 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0c3b8-6000-450d-bcf2-9a54b5bc7153 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:31:00.008456 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0c3b8-6000-450d-bcf2-9a54b5bc7153 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:31:00.008593 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0c3b8-6000-450d-bcf2-9a54b5bc7153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:31:00.029877 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-ada0c3b8-6000-450d-bcf2-9a54b5bc7153 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:31:00.031096 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ada0c3b8-6000-450d-bcf2-9a54b5bc7153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:31:00.039648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 763/1517] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 12:31:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:31:00.197325 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6aee21-469e-4763-b8b5-9412f553f5e6 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:31:00.216518 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6aee21-469e-4763-b8b5-9412f553f5e6 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:31:00.221984 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d827c-5f52-449b-8e22-db9eff6f2ee5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:31:00.222639 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d827c-5f52-449b-8e22-db9eff6f2ee5 service nova] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:31:00.222921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d827c-5f52-449b-8e22-db9eff6f2ee5 service nova] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:31:00.223197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d827c-5f52-449b-8e22-db9eff6f2ee5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:31:00.232395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d827c-5f52-449b-8e22-db9eff6f2ee5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:31:00.234028 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d827c-5f52-449b-8e22-db9eff6f2ee5 service nova] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:31:00.244156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 755/1518] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:31:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:31:03.335554 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9cfdf-5763-4847-bad4-d27a2b20ffa5 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:31:03.336091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9cfdf-5763-4847-bad4-d27a2b20ffa5 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:31:03.336091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9cfdf-5763-4847-bad4-d27a2b20ffa5 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:31:03.337486 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 764/1519] 10.222.0.16 () {54 vars in 960 bytes} [Mon May 23 12:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:31:03.343387 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6b061-7265-4303-8257-c40787f698d4 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:31:03.343908 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6b061-7265-4303-8257-c40787f698d4 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:31:03.344034 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6b061-7265-4303-8257-c40787f698d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:31:03.363494 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-e8b6b061-7265-4303-8257-c40787f698d4 None None] MFA Rules not processed for user `ebc0f137e0d24dd2a460689bc17a1b98`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:31:03.364871 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-e8b6b061-7265-4303-8257-c40787f698d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:31:03.374579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 756/1520] 10.222.0.16 () {58 vars in 1057 bytes} [Mon May 23 12:31:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:31:03.382130 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff9cfdf-5763-4847-bad4-d27a2b20ffa5 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:31:03.396717 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff9cfdf-5763-4847-bad4-d27a2b20ffa5 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:31:03.401775 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502f734-0ebe-482d-9a4e-0c3948f8800b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc0f137e0d24dd2a460689bc17a1b98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:31:03.402435 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502f734-0ebe-482d-9a4e-0c3948f8800b service placement] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:31:03.402722 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502f734-0ebe-482d-9a4e-0c3948f8800b service placement] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:31:03.403029 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502f734-0ebe-482d-9a4e-0c3948f8800b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:31:03.414322 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502f734-0ebe-482d-9a4e-0c3948f8800b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:31:03.415963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502f734-0ebe-482d-9a4e-0c3948f8800b service placement] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:31:03.427651 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 765/1521] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:31:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:31:11.591597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ed294-2b71-480f-93a2-5d9e48574bab None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:31:11.592190 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ed294-2b71-480f-93a2-5d9e48574bab None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:31:11.592190 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ed294-2b71-480f-93a2-5d9e48574bab None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:31:11.593553 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 757/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 12:31:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:31:11.598537 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c5ef7-540d-4cd3-943a-b5585ea37c20 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:31:11.598981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c5ef7-540d-4cd3-943a-b5585ea37c20 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:31:11.599086 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c5ef7-540d-4cd3-943a-b5585ea37c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:31:11.617961 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-683c5ef7-540d-4cd3-943a-b5585ea37c20 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:31:11.619155 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-683c5ef7-540d-4cd3-943a-b5585ea37c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:31:11.627357 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 766/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 12:31:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:31:11.783863 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ed294-2b71-480f-93a2-5d9e48574bab None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:31:11.799284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ed294-2b71-480f-93a2-5d9e48574bab None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:31:11.804727 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a83376-51e3-40eb-b5a2-533cfd193218 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:31:11.805380 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a83376-51e3-40eb-b5a2-533cfd193218 service nova] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:31:11.805689 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a83376-51e3-40eb-b5a2-533cfd193218 service nova] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:31:11.805969 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a83376-51e3-40eb-b5a2-533cfd193218 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:31:11.815013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a83376-51e3-40eb-b5a2-533cfd193218 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:31:11.816670 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a83376-51e3-40eb-b5a2-533cfd193218 service nova] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:31:11.826782 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 758/1524] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:31:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:31:12.732234 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85724b0-58fd-4e35-b344-e0de9ed1eb47 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:31:12.732828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85724b0-58fd-4e35-b344-e0de9ed1eb47 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:31:12.732828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85724b0-58fd-4e35-b344-e0de9ed1eb47 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:31:12.734172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 767/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 12:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:31:12.739014 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe5535-d3e4-457c-b240-329aa4a9c6b3 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:31:12.739467 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe5535-d3e4-457c-b240-329aa4a9c6b3 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:31:12.739563 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe5535-d3e4-457c-b240-329aa4a9c6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:31:12.759488 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-96fe5535-d3e4-457c-b240-329aa4a9c6b3 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:31:12.760737 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-96fe5535-d3e4-457c-b240-329aa4a9c6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:31:12.769430 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 759/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 12:31:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:31:12.922729 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85724b0-58fd-4e35-b344-e0de9ed1eb47 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:31:12.937975 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85724b0-58fd-4e35-b344-e0de9ed1eb47 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:31:12.943347 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b73515d-a9c1-4b0e-ad61-180732e667af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:31:12.943988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b73515d-a9c1-4b0e-ad61-180732e667af service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:31:12.944308 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b73515d-a9c1-4b0e-ad61-180732e667af service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:31:12.944591 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b73515d-a9c1-4b0e-ad61-180732e667af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:31:12.953460 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b73515d-a9c1-4b0e-ad61-180732e667af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:31:12.955072 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b73515d-a9c1-4b0e-ad61-180732e667af service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:31:12.964914 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 768/1527] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:31:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:31:17.447728 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335b8ee-11b0-4bc5-b4f1-a41921f51c7d None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:31:17.448341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335b8ee-11b0-4bc5-b4f1-a41921f51c7d None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:31:17.448341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335b8ee-11b0-4bc5-b4f1-a41921f51c7d None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:31:17.450277 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 760/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 12:31:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:31:17.455756 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7527137-b757-40be-898c-1820ab329672 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:31:17.456270 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7527137-b757-40be-898c-1820ab329672 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:31:17.456393 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7527137-b757-40be-898c-1820ab329672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:31:17.474837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-e7527137-b757-40be-898c-1820ab329672 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:31:17.476039 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e7527137-b757-40be-898c-1820ab329672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:31:17.484406 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 769/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 12:31:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:31:17.521487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335b8ee-11b0-4bc5-b4f1-a41921f51c7d None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:31:17.523187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-5335b8ee-11b0-4bc5-b4f1-a41921f51c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:31:17.582852 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335b8ee-11b0-4bc5-b4f1-a41921f51c7d None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:31:17.586418 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb386f67-1171-4c3b-ae3d-5f3122fc0974 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:31:17.587081 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb386f67-1171-4c3b-ae3d-5f3122fc0974 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:31:17.587369 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb386f67-1171-4c3b-ae3d-5f3122fc0974 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:31:17.587655 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb386f67-1171-4c3b-ae3d-5f3122fc0974 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:31:17.597660 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb386f67-1171-4c3b-ae3d-5f3122fc0974 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:31:17.599111 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb386f67-1171-4c3b-ae3d-5f3122fc0974 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:31:17.610463 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 761/1530] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:31:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:31:19.905969 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceea96b-328e-4281-be7d-d006fd469d14 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:31:19.906592 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceea96b-328e-4281-be7d-d006fd469d14 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:31:19.906592 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceea96b-328e-4281-be7d-d006fd469d14 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:31:19.907887 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 770/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 12:31:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:31:19.913416 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f31490-17d9-49bf-ba6f-e6d8594aed0e None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:31:19.913910 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f31490-17d9-49bf-ba6f-e6d8594aed0e None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:31:19.914035 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f31490-17d9-49bf-ba6f-e6d8594aed0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:31:19.932263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-15f31490-17d9-49bf-ba6f-e6d8594aed0e None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:31:19.934001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-15f31490-17d9-49bf-ba6f-e6d8594aed0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:31:19.941943 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 762/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 12:31:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:31:19.979232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceea96b-328e-4281-be7d-d006fd469d14 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:31:19.981413 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-aceea96b-328e-4281-be7d-d006fd469d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:31:19.989141 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=85 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:31:19.996204 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceea96b-328e-4281-be7d-d006fd469d14 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:31:19.999784 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eec00e-b05a-4b9c-ac5c-1c66dd1645f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:31:20.000452 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eec00e-b05a-4b9c-ac5c-1c66dd1645f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:31:20.000800 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eec00e-b05a-4b9c-ac5c-1c66dd1645f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:31:20.001082 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eec00e-b05a-4b9c-ac5c-1c66dd1645f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:31:20.010738 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eec00e-b05a-4b9c-ac5c-1c66dd1645f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:31:20.013060 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eec00e-b05a-4b9c-ac5c-1c66dd1645f3 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:31:20.023177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 771/1533] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:31:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:31:23.931706 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8ea38-7b75-434d-96d8-9df8123d5c55 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:31:23.932299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8ea38-7b75-434d-96d8-9df8123d5c55 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:31:23.932299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8ea38-7b75-434d-96d8-9df8123d5c55 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:31:23.934480 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 763/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 12:31:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:31:23.940235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983b970-8cf8-4010-b277-ab30c58199c2 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:31:23.941014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983b970-8cf8-4010-b277-ab30c58199c2 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:31:23.941014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983b970-8cf8-4010-b277-ab30c58199c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:31:23.960112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-7983b970-8cf8-4010-b277-ab30c58199c2 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:31:23.961381 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-7983b970-8cf8-4010-b277-ab30c58199c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:31:23.969594 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 772/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 12:31:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:31:24.306166 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8ea38-7b75-434d-96d8-9df8123d5c55 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:31:24.308063 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-efa8ea38-7b75-434d-96d8-9df8123d5c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:31:24.315031 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=75 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:31:24.390820 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8ea38-7b75-434d-96d8-9df8123d5c55 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:31:24.396235 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c88e1b1-a2dd-4a9e-9e63-47f28c0efc52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:31:24.396913 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88e1b1-a2dd-4a9e-9e63-47f28c0efc52 service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:31:24.397293 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88e1b1-a2dd-4a9e-9e63-47f28c0efc52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:31:24.397577 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c88e1b1-a2dd-4a9e-9e63-47f28c0efc52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:31:24.408289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c88e1b1-a2dd-4a9e-9e63-47f28c0efc52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:31:24.409771 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c88e1b1-a2dd-4a9e-9e63-47f28c0efc52 service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:31:24.424368 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 764/1536] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:31:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:31:26.830924 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf29c0-5ad3-4621-8961-d42d013e913c None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:31:26.831505 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf29c0-5ad3-4621-8961-d42d013e913c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:31:26.831505 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf29c0-5ad3-4621-8961-d42d013e913c None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:31:26.832916 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 773/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 12:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:31:26.838374 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62445bbe-bd5e-47ef-8ffb-37a31cc4c3ea None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:31:26.838876 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62445bbe-bd5e-47ef-8ffb-37a31cc4c3ea None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:31:26.838989 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62445bbe-bd5e-47ef-8ffb-37a31cc4c3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:31:26.859118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-62445bbe-bd5e-47ef-8ffb-37a31cc4c3ea None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:31:26.860385 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-62445bbe-bd5e-47ef-8ffb-37a31cc4c3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:31:26.868244 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 765/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 12:31:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:31:27.145778 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf29c0-5ad3-4621-8961-d42d013e913c None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:31:27.147570 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-afaf29c0-5ad3-4621-8961-d42d013e913c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:31:27.161218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf29c0-5ad3-4621-8961-d42d013e913c None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:31:27.164925 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb323b36-9c64-4a0a-8246-0b59d4a7ec07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:31:27.165575 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb323b36-9c64-4a0a-8246-0b59d4a7ec07 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:31:27.165862 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb323b36-9c64-4a0a-8246-0b59d4a7ec07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:31:27.166142 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb323b36-9c64-4a0a-8246-0b59d4a7ec07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:31:27.175600 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb323b36-9c64-4a0a-8246-0b59d4a7ec07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:31:27.177061 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb323b36-9c64-4a0a-8246-0b59d4a7ec07 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:31:27.187256 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 774/1539] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:31:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:31:36.852835 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=28 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:31:37.182653 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=14 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:01.987008 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2aac21-da87-4e22-abea-eab8f4d370ba None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:01.987643 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2aac21-da87-4e22-abea-eab8f4d370ba None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:01.987643 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2aac21-da87-4e22-abea-eab8f4d370ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.018189 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-4b2aac21-da87-4e22-abea-eab8f4d370ba None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:02.019424 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-4b2aac21-da87-4e22-abea-eab8f4d370ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:02.038100 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 766/1540] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:02.062357 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9e125-c90f-4abd-b8ed-b03f764e3b7a None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:02.063223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9e125-c90f-4abd-b8ed-b03f764e3b7a None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:02.063223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c9e125-c90f-4abd-b8ed-b03f764e3b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.092123 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-13c9e125-c90f-4abd-b8ed-b03f764e3b7a None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:02.093408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-13c9e125-c90f-4abd-b8ed-b03f764e3b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:02.095412 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07e1e6-0f39-4e8d-8af3-a96e5e17f844 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:02.095816 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07e1e6-0f39-4e8d-8af3-a96e5e17f844 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:02.096355 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07e1e6-0f39-4e8d-8af3-a96e5e17f844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.102269 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 775/1541] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:02.109245 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c9e125-c90f-4abd-b8ed-b03f764e3b7a None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:02.118958 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c9e125-c90f-4abd-b8ed-b03f764e3b7a None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:02.129388 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-3e07e1e6-0f39-4e8d-8af3-a96e5e17f844 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:02.129627 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e80a73-73a7-4166-80c7-c919738a1ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:02.130161 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e80a73-73a7-4166-80c7-c919738a1ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:02.130459 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e80a73-73a7-4166-80c7-c919738a1ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:02.130741 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e80a73-73a7-4166-80c7-c919738a1ba2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.131503 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e80a73-73a7-4166-80c7-c919738a1ba2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:33:02.131815 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e80a73-73a7-4166-80c7-c919738a1ba2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:02.132916 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: WARNING py.warnings [None req-25e80a73-73a7-4166-80c7-c919738a1ba2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 12:33:02.132916 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: warnings.warn(msg) May 23 12:33:02.132916 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: May 23 12:33:02.133839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e80a73-73a7-4166-80c7-c919738a1ba2 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:02.134455 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-3e07e1e6-0f39-4e8d-8af3-a96e5e17f844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:02.153059 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 776/1542] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:33:02.154560 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e2d3b-f4c4-44a9-9895-9246c805c148 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:02.154856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e2d3b-f4c4-44a9-9895-9246c805c148 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:02.155142 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e2d3b-f4c4-44a9-9895-9246c805c148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.160385 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 767/1543] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:33:02.173686 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07e1e6-0f39-4e8d-8af3-a96e5e17f844 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:02.187167 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-ba3e2d3b-f4c4-44a9-9895-9246c805c148 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:02.188497 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ba3e2d3b-f4c4-44a9-9895-9246c805c148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:02.195509 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07e1e6-0f39-4e8d-8af3-a96e5e17f844 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:02.207202 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d41350-9baf-4582-a154-b2d2112af18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:02.208353 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d41350-9baf-4582-a154-b2d2112af18d admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:02.208899 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d41350-9baf-4582-a154-b2d2112af18d admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:02.209395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d41350-9baf-4582-a154-b2d2112af18d admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.210331 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d41350-9baf-4582-a154-b2d2112af18d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:02.211625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: WARNING py.warnings [None req-c7d41350-9baf-4582-a154-b2d2112af18d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 12:33:02.211625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: warnings.warn(msg) May 23 12:33:02.211625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: May 23 12:33:02.213305 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d41350-9baf-4582-a154-b2d2112af18d admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:02.220200 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 777/1544] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:02.220200 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b38bf9-88fa-4953-8522-ede9908f9369 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:02.220200 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b38bf9-88fa-4953-8522-ede9908f9369 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:02.220200 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b38bf9-88fa-4953-8522-ede9908f9369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.251844 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-34b38bf9-88fa-4953-8522-ede9908f9369 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:02.252865 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-34b38bf9-88fa-4953-8522-ede9908f9369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:02.278596 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 778/1545] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:02.280130 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a2310-d273-46da-a1da-31f2165e91fe None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:02.280456 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a2310-d273-46da-a1da-31f2165e91fe None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:02.280744 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a2310-d273-46da-a1da-31f2165e91fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.294268 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 768/1546] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:33:02 2022] POST /identity/v3/projects => generated 401 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:02.296342 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9860bf-a51e-4a9c-bd1a-421f0189b984 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:02.296907 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9860bf-a51e-4a9c-bd1a-421f0189b984 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:02.297483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9860bf-a51e-4a9c-bd1a-421f0189b984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.319972 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-366a2310-d273-46da-a1da-31f2165e91fe None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:02.321244 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-366a2310-d273-46da-a1da-31f2165e91fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:02.332618 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-6a9860bf-a51e-4a9c-bd1a-421f0189b984 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:02.334328 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-6a9860bf-a51e-4a9c-bd1a-421f0189b984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:02.400372 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 779/1547] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:02.402985 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a2310-d273-46da-a1da-31f2165e91fe None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:02.420915 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a2310-d273-46da-a1da-31f2165e91fe None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:02.427511 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e72ea0-97bc-4599-a37b-ea0c8c6b3ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:02.429421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e72ea0-97bc-4599-a37b-ea0c8c6b3ceb admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:02.429421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e72ea0-97bc-4599-a37b-ea0c8c6b3ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:02.429421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e72ea0-97bc-4599-a37b-ea0c8c6b3ceb admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.431556 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e72ea0-97bc-4599-a37b-ea0c8c6b3ceb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:33:02.431889 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e72ea0-97bc-4599-a37b-ea0c8c6b3ceb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:02.434624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e72ea0-97bc-4599-a37b-ea0c8c6b3ceb admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:02.439895 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 769/1548] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:02.441459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9860bf-a51e-4a9c-bd1a-421f0189b984 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:02.450841 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 780/1549] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:33:02.452208 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d529b-9d1a-427b-a830-02f838189fa7 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:02.452502 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d529b-9d1a-427b-a830-02f838189fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:02.452786 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d529b-9d1a-427b-a830-02f838189fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.454524 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9860bf-a51e-4a9c-bd1a-421f0189b984 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:02.459959 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d62656-4d54-41de-bbe4-30d8bffb410b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:02.461412 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d62656-4d54-41de-bbe4-30d8bffb410b admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:02.461931 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d62656-4d54-41de-bbe4-30d8bffb410b admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:02.462777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d62656-4d54-41de-bbe4-30d8bffb410b admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.464520 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d62656-4d54-41de-bbe4-30d8bffb410b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:02.466758 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d62656-4d54-41de-bbe4-30d8bffb410b admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:02.476157 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-c21d529b-9d1a-427b-a830-02f838189fa7 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:02.477538 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c21d529b-9d1a-427b-a830-02f838189fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:02.491812 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 781/1550] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:02.491812 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a869df8-0acc-49d0-a9c3-067ae7f33dd4 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:02.491812 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a869df8-0acc-49d0-a9c3-067ae7f33dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:02.491812 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a869df8-0acc-49d0-a9c3-067ae7f33dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.499874 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 770/1551] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:33:02 2022] POST /identity/v3/users => generated 301 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:02.502784 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39541caf-5834-4a68-becd-773070de7642 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:02.503169 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39541caf-5834-4a68-becd-773070de7642 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:02.503711 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39541caf-5834-4a68-becd-773070de7642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.508695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-0a869df8-0acc-49d0-a9c3-067ae7f33dd4 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:02.509908 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-0a869df8-0acc-49d0-a9c3-067ae7f33dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:02.520216 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 782/1552] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:02.521032 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a869df8-0acc-49d0-a9c3-067ae7f33dd4 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:02.524754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-39541caf-5834-4a68-becd-773070de7642 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:02.526631 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-39541caf-5834-4a68-becd-773070de7642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:02.531726 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a869df8-0acc-49d0-a9c3-067ae7f33dd4 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:02.536405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 771/1553] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:02.537224 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39541caf-5834-4a68-becd-773070de7642 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:02.538608 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20dc67-b4a8-4c9c-bd74-db29107cc009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:02.539780 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20dc67-b4a8-4c9c-bd74-db29107cc009 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:02.540649 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20dc67-b4a8-4c9c-bd74-db29107cc009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:02.540978 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20dc67-b4a8-4c9c-bd74-db29107cc009 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.542688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20dc67-b4a8-4c9c-bd74-db29107cc009 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:33:02.543079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20dc67-b4a8-4c9c-bd74-db29107cc009 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:02.545195 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20dc67-b4a8-4c9c-bd74-db29107cc009 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:02.549099 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39541caf-5834-4a68-becd-773070de7642 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:02.555398 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005ddb2-08b6-465e-9594-18bfa24fe3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:02.556077 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005ddb2-08b6-465e-9594-18bfa24fe3ef admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:02.556562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005ddb2-08b6-465e-9594-18bfa24fe3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:02.557380 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9005ddb2-08b6-465e-9594-18bfa24fe3ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.564745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 783/1554] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:33:02.564947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9005ddb2-08b6-465e-9594-18bfa24fe3ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:02.565049 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd3178-3fea-446f-b326-84e733ecf41e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:02.565049 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd3178-3fea-446f-b326-84e733ecf41e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:02.565049 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd3178-3fea-446f-b326-84e733ecf41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.565279 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9005ddb2-08b6-465e-9594-18bfa24fe3ef admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:02.584641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-5bdd3178-3fea-446f-b326-84e733ecf41e None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:02.585873 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5bdd3178-3fea-446f-b326-84e733ecf41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:02.595403 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 784/1555] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:02.596725 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f33dc-0a2f-44f8-9fb5-9767446d62b9 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:02.597019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f33dc-0a2f-44f8-9fb5-9767446d62b9 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:02.597299 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f33dc-0a2f-44f8-9fb5-9767446d62b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.600109 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 772/1556] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:33:02 2022] POST /identity/v3/projects => generated 377 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:02.600718 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005ddb2-08b6-465e-9594-18bfa24fe3ef admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:02.621848 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-da8f33dc-0a2f-44f8-9fb5-9767446d62b9 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:02.623176 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-da8f33dc-0a2f-44f8-9fb5-9767446d62b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:02.686187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9005ddb2-08b6-465e-9594-18bfa24fe3ef admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:02.690867 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44fdc2-da47-4a36-a15f-8f64313c0d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:02.691546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44fdc2-da47-4a36-a15f-8f64313c0d28 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:02.691884 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44fdc2-da47-4a36-a15f-8f64313c0d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:02.692190 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44fdc2-da47-4a36-a15f-8f64313c0d28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.692932 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d44fdc2-da47-4a36-a15f-8f64313c0d28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:02.694578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d44fdc2-da47-4a36-a15f-8f64313c0d28 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:02.702582 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 785/1557] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:02.705837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ea66e-f75d-4501-82d0-abf2a561652f None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:02.706384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ea66e-f75d-4501-82d0-abf2a561652f None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:02.706496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 773/1558] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:02.706944 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ea66e-f75d-4501-82d0-abf2a561652f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.707757 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b977b43-6a71-4839-a0d3-21f9954b4c96 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:02.708054 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b977b43-6a71-4839-a0d3-21f9954b4c96 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:02.708377 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b977b43-6a71-4839-a0d3-21f9954b4c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.729660 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-7b977b43-6a71-4839-a0d3-21f9954b4c96 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:02.731069 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-7b977b43-6a71-4839-a0d3-21f9954b4c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:02.735675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-bd5ea66e-f75d-4501-82d0-abf2a561652f None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:02.736897 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-bd5ea66e-f75d-4501-82d0-abf2a561652f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:02.739076 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 774/1559] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:02.743916 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b977b43-6a71-4839-a0d3-21f9954b4c96 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:02.745205 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 786/1560] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:02.746004 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ea66e-f75d-4501-82d0-abf2a561652f None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:02.754338 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b977b43-6a71-4839-a0d3-21f9954b4c96 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:02.755550 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ea66e-f75d-4501-82d0-abf2a561652f None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:02.759253 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28839a25-8bea-4301-9f98-a82a49834e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:02.759949 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28839a25-8bea-4301-9f98-a82a49834e49 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:02.760333 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb129c25-9e0d-46b6-b553-6389c03aa84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:02.760457 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28839a25-8bea-4301-9f98-a82a49834e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:02.760900 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28839a25-8bea-4301-9f98-a82a49834e49 admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.761051 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb129c25-9e0d-46b6-b553-6389c03aa84e admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:02.761463 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb129c25-9e0d-46b6-b553-6389c03aa84e admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:02.761746 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb129c25-9e0d-46b6-b553-6389c03aa84e admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.761854 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28839a25-8bea-4301-9f98-a82a49834e49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:33:02.762067 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28839a25-8bea-4301-9f98-a82a49834e49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:02.762440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb129c25-9e0d-46b6-b553-6389c03aa84e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:02.763706 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28839a25-8bea-4301-9f98-a82a49834e49 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:02.764441 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb129c25-9e0d-46b6-b553-6389c03aa84e admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:02.767134 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:02.786206 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 775/1561] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:33:02.786206 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28839a25-8bea-4301-9f98-a82a49834e49 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:02.786537 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:02.791802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28839a25-8bea-4301-9f98-a82a49834e49 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:02.799260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 787/1562] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:33:02 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:02.799473 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9030ec-1121-4c57-b157-b17c031ed9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:02.799473 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9030ec-1121-4c57-b157-b17c031ed9f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:02.799669 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009dfd8-4a45-4df4-a8ba-ffc17e36f827 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:02.799780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9030ec-1121-4c57-b157-b17c031ed9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:02.799878 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009dfd8-4a45-4df4-a8ba-ffc17e36f827 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:02.799976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9030ec-1121-4c57-b157-b17c031ed9f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.800071 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009dfd8-4a45-4df4-a8ba-ffc17e36f827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.800195 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9030ec-1121-4c57-b157-b17c031ed9f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:02.800480 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9030ec-1121-4c57-b157-b17c031ed9f8 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:02.821062 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-6009dfd8-4a45-4df4-a8ba-ffc17e36f827 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:02.822764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-6009dfd8-4a45-4df4-a8ba-ffc17e36f827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:02.828287 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 776/1563] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:33:02 2022] POST /identity/v3/users => generated 289 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:02.830437 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c0743-7be3-45b7-b842-5f5d130c5ef1 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:02.831012 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c0743-7be3-45b7-b842-5f5d130c5ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:02.831493 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c0743-7be3-45b7-b842-5f5d130c5ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.856074 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-273c0743-7be3-45b7-b842-5f5d130c5ef1 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:02.857341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-273c0743-7be3-45b7-b842-5f5d130c5ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:02.890483 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 788/1564] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:02.891345 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009dfd8-4a45-4df4-a8ba-ffc17e36f827 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:02.901041 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009dfd8-4a45-4df4-a8ba-ffc17e36f827 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:02.905778 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41275c7a-1ad9-48da-8b98-f677e768cabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:02.906580 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41275c7a-1ad9-48da-8b98-f677e768cabb admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:02.906958 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41275c7a-1ad9-48da-8b98-f677e768cabb admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:02.907353 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41275c7a-1ad9-48da-8b98-f677e768cabb admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.908195 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41275c7a-1ad9-48da-8b98-f677e768cabb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:02.909950 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41275c7a-1ad9-48da-8b98-f677e768cabb admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:02.924041 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 789/1565] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:02.924041 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41275c7a-1ad9-48da-8b98-f677e768cabb admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:02.930153 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 777/1566] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:02.931089 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c0743-7be3-45b7-b842-5f5d130c5ef1 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:02.931760 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41275c7a-1ad9-48da-8b98-f677e768cabb admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:02.938099 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ad7d6-c73d-4606-b1d6-c283e4786b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:02.938776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ad7d6-c73d-4606-b1d6-c283e4786b61 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:02.939060 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ad7d6-c73d-4606-b1d6-c283e4786b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:02.939338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ad7d6-c73d-4606-b1d6-c283e4786b61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.940060 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ad7d6-c73d-4606-b1d6-c283e4786b61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:33:02.940403 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ad7d6-c73d-4606-b1d6-c283e4786b61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:02.941990 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c0743-7be3-45b7-b842-5f5d130c5ef1 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:02.942104 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ad7d6-c73d-4606-b1d6-c283e4786b61 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:02.947496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f9a78-6383-426a-b43d-bf7e4ba4a35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:02.948519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f9a78-6383-426a-b43d-bf7e4ba4a35e admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:02.948818 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f9a78-6383-426a-b43d-bf7e4ba4a35e admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:02.949852 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f9a78-6383-426a-b43d-bf7e4ba4a35e admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.950101 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f9a78-6383-426a-b43d-bf7e4ba4a35e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:33:02.950411 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f9a78-6383-426a-b43d-bf7e4ba4a35e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:02.952053 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f9a78-6383-426a-b43d-bf7e4ba4a35e admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:02.955335 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 790/1567] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:33:02.956384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ad7d6-c73d-4606-b1d6-c283e4786b61 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:02.968037 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ad7d6-c73d-4606-b1d6-c283e4786b61 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:02.968901 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 778/1568] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:33:02.972950 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5ad70-5d85-4735-bd36-575c817b35b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:02.973618 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5ad70-5d85-4735-bd36-575c817b35b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:02.973903 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5ad70-5d85-4735-bd36-575c817b35b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:02.974180 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5ad70-5d85-4735-bd36-575c817b35b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.974871 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5ad70-5d85-4735-bd36-575c817b35b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:02.976817 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5ad70-5d85-4735-bd36-575c817b35b8 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:02.976949 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f9a78-6383-426a-b43d-bf7e4ba4a35e admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:02.987543 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f9a78-6383-426a-b43d-bf7e4ba4a35e admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:02.992466 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ebadc-ad03-4853-8f7c-b18dab5093d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:02.993292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ebadc-ad03-4853-8f7c-b18dab5093d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:02.993748 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ebadc-ad03-4853-8f7c-b18dab5093d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:02.994060 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ebadc-ad03-4853-8f7c-b18dab5093d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:02.995570 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ebadc-ad03-4853-8f7c-b18dab5093d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:02.997869 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ebadc-ad03-4853-8f7c-b18dab5093d4 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:03.011650 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 791/1569] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:33:02 2022] POST /identity/v3/projects => generated 393 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:03.011650 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5ad70-5d85-4735-bd36-575c817b35b8 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:03.023280 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 779/1570] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:33:02 2022] POST /identity/v3/users => generated 294 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:03.023280 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ebadc-ad03-4853-8f7c-b18dab5093d4 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:03.033182 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ebadc-ad03-4853-8f7c-b18dab5093d4 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:03.076472 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5ad70-5d85-4735-bd36-575c817b35b8 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:03.081676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c57940-cbaf-48ed-9639-bb4e3b331c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:03.082366 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c57940-cbaf-48ed-9639-bb4e3b331c87 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:03.082653 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c57940-cbaf-48ed-9639-bb4e3b331c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:03.082941 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c57940-cbaf-48ed-9639-bb4e3b331c87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:03.083675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c57940-cbaf-48ed-9639-bb4e3b331c87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:03.085459 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c57940-cbaf-48ed-9639-bb4e3b331c87 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:03.093466 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccceec0-ea46-467f-a9d6-8103343fc703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:03.094148 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccceec0-ea46-467f-a9d6-8103343fc703 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:03.094434 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccceec0-ea46-467f-a9d6-8103343fc703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:03.094718 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccceec0-ea46-467f-a9d6-8103343fc703 admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:03.095442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccceec0-ea46-467f-a9d6-8103343fc703 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:33:03.095750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccceec0-ea46-467f-a9d6-8103343fc703 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:03.095850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 792/1571] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:03.096800 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c57940-cbaf-48ed-9639-bb4e3b331c87 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:03.097418 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccceec0-ea46-467f-a9d6-8103343fc703 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:03.117190 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 780/1572] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:33:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:33:03.117422 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c57940-cbaf-48ed-9639-bb4e3b331c87 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:03.117554 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccceec0-ea46-467f-a9d6-8103343fc703 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:03.117657 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaba88f-85bc-4a89-88a8-2080ff78ca09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:03.117657 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaba88f-85bc-4a89-88a8-2080ff78ca09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:03.117657 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaba88f-85bc-4a89-88a8-2080ff78ca09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:03.117657 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaba88f-85bc-4a89-88a8-2080ff78ca09 admin admin] PATH_INFO: `/v3/projects/abbd5b0e8cf742cfa1429996026462d4/users/04f89ee7b66247d28c44ffbad8b119f8/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:03.121801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccceec0-ea46-467f-a9d6-8103343fc703 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:03.127584 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6b898-9476-42e3-b8c1-607194c9bb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:03.128281 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6b898-9476-42e3-b8c1-607194c9bb1c admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:03.128756 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6b898-9476-42e3-b8c1-607194c9bb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:03.129037 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6b898-9476-42e3-b8c1-607194c9bb1c admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:03.129771 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e6b898-9476-42e3-b8c1-607194c9bb1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:33:03.130082 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e6b898-9476-42e3-b8c1-607194c9bb1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:03.131683 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e6b898-9476-42e3-b8c1-607194c9bb1c admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:03.133966 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaba88f-85bc-4a89-88a8-2080ff78ca09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abbd5b0e8cf742cfa1429996026462d4, user_id=04f89ee7b66247d28c44ffbad8b119f8, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:03.136892 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaba88f-85bc-4a89-88a8-2080ff78ca09 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:03.145202 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 781/1573] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:33:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:33:03.149139 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6b898-9476-42e3-b8c1-607194c9bb1c admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:03.151663 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 793/1574] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:03 2022] PUT /identity/v3/projects/abbd5b0e8cf742cfa1429996026462d4/users/04f89ee7b66247d28c44ffbad8b119f8/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:03.157307 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaba88f-85bc-4a89-88a8-2080ff78ca09 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:03.160645 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6b898-9476-42e3-b8c1-607194c9bb1c admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:03.183233 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:03.195727 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:03.230525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaba88f-85bc-4a89-88a8-2080ff78ca09 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:03.230783 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15799d15-849d-4e9c-974b-2596b95a1a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:03.231277 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15799d15-849d-4e9c-974b-2596b95a1a2b admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:03.231639 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15799d15-849d-4e9c-974b-2596b95a1a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:03.231845 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15799d15-849d-4e9c-974b-2596b95a1a2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:03.232576 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15799d15-849d-4e9c-974b-2596b95a1a2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:03.234491 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15799d15-849d-4e9c-974b-2596b95a1a2b admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:03.236502 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee8c00-6a9d-450b-ab7f-1f15886c54ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:03.237171 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee8c00-6a9d-450b-ab7f-1f15886c54ae admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:03.237490 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee8c00-6a9d-450b-ab7f-1f15886c54ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:03.237772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee8c00-6a9d-450b-ab7f-1f15886c54ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:03.238721 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee8c00-6a9d-450b-ab7f-1f15886c54ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:03.241326 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ee8c00-6a9d-450b-ab7f-1f15886c54ae admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:03.266453 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 782/1575] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:33:03 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:03.266453 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15799d15-849d-4e9c-974b-2596b95a1a2b admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:03.277306 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 794/1576] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:33:03 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:03.277306 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee8c00-6a9d-450b-ab7f-1f15886c54ae admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:03.334430 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15799d15-849d-4e9c-974b-2596b95a1a2b admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:03.340483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f49451-b37d-4e66-b1f2-0281a8b1de95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:03.341216 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f49451-b37d-4e66-b1f2-0281a8b1de95 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:03.341513 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f49451-b37d-4e66-b1f2-0281a8b1de95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:03.341818 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f49451-b37d-4e66-b1f2-0281a8b1de95 admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:03.342528 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f49451-b37d-4e66-b1f2-0281a8b1de95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:03.342647 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee8c00-6a9d-450b-ab7f-1f15886c54ae admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:03.344587 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f49451-b37d-4e66-b1f2-0281a8b1de95 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:03.348373 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc922010-b4dd-4f0f-a163-6fe235af96aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:03.349054 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc922010-b4dd-4f0f-a163-6fe235af96aa admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:03.349338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc922010-b4dd-4f0f-a163-6fe235af96aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:03.349603 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc922010-b4dd-4f0f-a163-6fe235af96aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:03.350332 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc922010-b4dd-4f0f-a163-6fe235af96aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:03.351930 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc922010-b4dd-4f0f-a163-6fe235af96aa admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:03.368511 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 795/1577] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:03.369291 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc922010-b4dd-4f0f-a163-6fe235af96aa admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:03.369802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 783/1578] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:33:03 2022] POST /identity/v3/users => generated 297 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:03.370703 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f49451-b37d-4e66-b1f2-0281a8b1de95 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:03.381235 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f49451-b37d-4e66-b1f2-0281a8b1de95 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:03.383438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc922010-b4dd-4f0f-a163-6fe235af96aa admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:03.386191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfddbb6-e625-4d54-abbc-6c5ba324db10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:03.386892 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfddbb6-e625-4d54-abbc-6c5ba324db10 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:03.387152 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfddbb6-e625-4d54-abbc-6c5ba324db10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:03.387438 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfddbb6-e625-4d54-abbc-6c5ba324db10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:03.388134 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfddbb6-e625-4d54-abbc-6c5ba324db10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:03.389424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da21543-6cd0-4e63-8cb4-b67928458adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:03.390082 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfddbb6-e625-4d54-abbc-6c5ba324db10 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:03.390224 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da21543-6cd0-4e63-8cb4-b67928458adf admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:03.390396 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da21543-6cd0-4e63-8cb4-b67928458adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:03.390650 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da21543-6cd0-4e63-8cb4-b67928458adf admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:03.391363 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da21543-6cd0-4e63-8cb4-b67928458adf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:03.392961 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da21543-6cd0-4e63-8cb4-b67928458adf admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:03.409041 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 796/1579] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:03.409041 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da21543-6cd0-4e63-8cb4-b67928458adf admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:03.418128 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da21543-6cd0-4e63-8cb4-b67928458adf admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:03.429680 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 784/1580] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:33:03 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:03.429951 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb1d9c-b041-4a48-8644-61e640dca6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:03.429951 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb1d9c-b041-4a48-8644-61e640dca6fb admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:03.429951 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb1d9c-b041-4a48-8644-61e640dca6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:03.429951 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb1d9c-b041-4a48-8644-61e640dca6fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:03.430672 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb1d9c-b041-4a48-8644-61e640dca6fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:03.430672 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb1d9c-b041-4a48-8644-61e640dca6fb admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:03.432739 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059fb9e6-6861-409c-a86d-22cd8b212abc None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:03.433180 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059fb9e6-6861-409c-a86d-22cd8b212abc None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:03.433562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059fb9e6-6861-409c-a86d-22cd8b212abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:03.468345 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 797/1581] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:33:03 2022] POST /identity/v3/projects => generated 365 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:03.469453 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb1d9c-b041-4a48-8644-61e640dca6fb admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:03.472980 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-059fb9e6-6861-409c-a86d-22cd8b212abc None None] MFA Rules not processed for user `04f89ee7b66247d28c44ffbad8b119f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:03.474235 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-059fb9e6-6861-409c-a86d-22cd8b212abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:03.541051 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb1d9c-b041-4a48-8644-61e640dca6fb admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:03.546557 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb026fd3-065b-4ba6-8c19-9b597c5c2c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:03.547263 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb026fd3-065b-4ba6-8c19-9b597c5c2c26 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:03.547547 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb026fd3-065b-4ba6-8c19-9b597c5c2c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:03.547823 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb026fd3-065b-4ba6-8c19-9b597c5c2c26 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:03.548548 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb026fd3-065b-4ba6-8c19-9b597c5c2c26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:03.549567 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: WARNING py.warnings [None req-bb026fd3-065b-4ba6-8c19-9b597c5c2c26 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 12:33:03.549567 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: warnings.warn(msg) May 23 12:33:03.549567 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: May 23 12:33:03.550791 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb026fd3-065b-4ba6-8c19-9b597c5c2c26 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:03.559366 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 785/1582] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:03 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:03.559366 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059fb9e6-6861-409c-a86d-22cd8b212abc None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:03.566610 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:03.570999 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059fb9e6-6861-409c-a86d-22cd8b212abc None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:03.576759 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 798/1583] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:33:03 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:03.576759 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb026fd3-065b-4ba6-8c19-9b597c5c2c26 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:03.577292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ead0a-7452-4867-8ac0-b64192f78ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:03.577960 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ead0a-7452-4867-8ac0-b64192f78ec4 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:03.578245 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ead0a-7452-4867-8ac0-b64192f78ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:03.578524 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ead0a-7452-4867-8ac0-b64192f78ec4 admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:03.579211 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ead0a-7452-4867-8ac0-b64192f78ec4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:03.581177 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ead0a-7452-4867-8ac0-b64192f78ec4 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:03.586431 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb026fd3-065b-4ba6-8c19-9b597c5c2c26 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:03.590983 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:03.592076 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6087b0-bdc4-4426-81d0-e218088ac735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:03.592896 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6087b0-bdc4-4426-81d0-e218088ac735 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:03.593211 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6087b0-bdc4-4426-81d0-e218088ac735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:03.593545 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6087b0-bdc4-4426-81d0-e218088ac735 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:03.594514 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6087b0-bdc4-4426-81d0-e218088ac735 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:03.596424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6087b0-bdc4-4426-81d0-e218088ac735 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:03.614684 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 799/1584] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:03.614684 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6087b0-bdc4-4426-81d0-e218088ac735 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:03.615003 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 786/1585] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:33:03 2022] POST /identity/v3/users => generated 289 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:03.615841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ead0a-7452-4867-8ac0-b64192f78ec4 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:03.624917 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6087b0-bdc4-4426-81d0-e218088ac735 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:03.626639 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ead0a-7452-4867-8ac0-b64192f78ec4 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:03.629850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb470fc7-ae54-4076-a140-408f11a170da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:03.630525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb470fc7-ae54-4076-a140-408f11a170da admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:03.630809 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb470fc7-ae54-4076-a140-408f11a170da admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:03.631087 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb470fc7-ae54-4076-a140-408f11a170da admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:03.631538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7338c-d358-44e4-970b-70fd75885a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:03.631990 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb470fc7-ae54-4076-a140-408f11a170da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:03.632525 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7338c-d358-44e4-970b-70fd75885a76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:03.632815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7338c-d358-44e4-970b-70fd75885a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:03.633099 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7338c-d358-44e4-970b-70fd75885a76 admin admin] PATH_INFO: `/v3/projects/1f94aad6b61d49da817d70d2b91a7594/users/f2936fe7026046aa9c43f1b1b0cb1bfd/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:03.633561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb470fc7-ae54-4076-a140-408f11a170da admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:03.647395 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 800/1586] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:03.648284 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb470fc7-ae54-4076-a140-408f11a170da admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:03.653226 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a7338c-d358-44e4-970b-70fd75885a76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f94aad6b61d49da817d70d2b91a7594, user_id=f2936fe7026046aa9c43f1b1b0cb1bfd, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:03.654341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: WARNING py.warnings [None req-99a7338c-d358-44e4-970b-70fd75885a76 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 12:33:03.654341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: warnings.warn(msg) May 23 12:33:03.654341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: May 23 12:33:03.656409 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a7338c-d358-44e4-970b-70fd75885a76 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:03.659382 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb470fc7-ae54-4076-a140-408f11a170da admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:03.664382 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bdd771-913f-45ed-abdd-febba52e60bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:03.665373 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdd771-913f-45ed-abdd-febba52e60bc admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:03.665663 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdd771-913f-45ed-abdd-febba52e60bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:03.665935 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdd771-913f-45ed-abdd-febba52e60bc admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:03.667455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bdd771-913f-45ed-abdd-febba52e60bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:03.669560 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bdd771-913f-45ed-abdd-febba52e60bc admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:03.670790 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 787/1587] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:03 2022] PUT /identity/v3/projects/1f94aad6b61d49da817d70d2b91a7594/users/f2936fe7026046aa9c43f1b1b0cb1bfd/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:03.671633 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7338c-d358-44e4-970b-70fd75885a76 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:03.689924 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 801/1588] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:33:03 2022] POST /identity/v3/users => generated 283 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:03.693765 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bdd771-913f-45ed-abdd-febba52e60bc admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:03.704588 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bdd771-913f-45ed-abdd-febba52e60bc admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:03.743979 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7338c-d358-44e4-970b-70fd75885a76 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:03.749028 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05577d-ad60-443b-8020-b0ed8a7be5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:03.749704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05577d-ad60-443b-8020-b0ed8a7be5ec admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:03.749991 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05577d-ad60-443b-8020-b0ed8a7be5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:03.750269 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05577d-ad60-443b-8020-b0ed8a7be5ec admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:03.750961 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea05577d-ad60-443b-8020-b0ed8a7be5ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:03.752935 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea05577d-ad60-443b-8020-b0ed8a7be5ec admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:03.769904 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8efea6c-631b-4d5c-b5a6-5b213a125a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:03.770576 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8efea6c-631b-4d5c-b5a6-5b213a125a33 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:03.770869 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8efea6c-631b-4d5c-b5a6-5b213a125a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:03.771148 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8efea6c-631b-4d5c-b5a6-5b213a125a33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:03.772114 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8efea6c-631b-4d5c-b5a6-5b213a125a33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:03.774375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8efea6c-631b-4d5c-b5a6-5b213a125a33 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:03.782402 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 788/1589] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:33:03 2022] POST /identity/v3/users => generated 283 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:03.783393 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05577d-ad60-443b-8020-b0ed8a7be5ec admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:03.794976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05577d-ad60-443b-8020-b0ed8a7be5ec admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:03.800377 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5bbbeb-9f38-4a64-aad5-9c823410ebd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:03.801062 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bbbeb-9f38-4a64-aad5-9c823410ebd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:03.801411 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bbbeb-9f38-4a64-aad5-9c823410ebd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:03.801697 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5bbbeb-9f38-4a64-aad5-9c823410ebd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:03.802496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5bbbeb-9f38-4a64-aad5-9c823410ebd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:03.804129 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5bbbeb-9f38-4a64-aad5-9c823410ebd3 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:03.806170 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 802/1590] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:33:03 2022] POST /identity/v3/projects => generated 401 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:03.809113 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8efea6c-631b-4d5c-b5a6-5b213a125a33 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:03.815480 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 789/1591] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:03.819380 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5bbbeb-9f38-4a64-aad5-9c823410ebd3 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:03.830724 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5bbbeb-9f38-4a64-aad5-9c823410ebd3 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:03.878106 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8efea6c-631b-4d5c-b5a6-5b213a125a33 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:03.883362 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51a5f3-f815-4b9b-8e0d-d0b71a2a7afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:03.884042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51a5f3-f815-4b9b-8e0d-d0b71a2a7afe admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:03.884366 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51a5f3-f815-4b9b-8e0d-d0b71a2a7afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:03.884652 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51a5f3-f815-4b9b-8e0d-d0b71a2a7afe admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:03.885940 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad51a5f3-f815-4b9b-8e0d-d0b71a2a7afe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:03.887557 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad51a5f3-f815-4b9b-8e0d-d0b71a2a7afe admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:03.891247 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb8b524-ff02-4f8b-aae6-b95edb7628f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:03.891922 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8b524-ff02-4f8b-aae6-b95edb7628f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:03.892233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8b524-ff02-4f8b-aae6-b95edb7628f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:03.892521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8b524-ff02-4f8b-aae6-b95edb7628f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:03.893247 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb8b524-ff02-4f8b-aae6-b95edb7628f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:03.894851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb8b524-ff02-4f8b-aae6-b95edb7628f2 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:03.897600 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 803/1592] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:03.898378 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51a5f3-f815-4b9b-8e0d-d0b71a2a7afe admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:03.905495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 790/1593] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:03.906523 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4c31d-6cc4-4dd0-b20a-1edfa20d715e None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:03.906902 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4c31d-6cc4-4dd0-b20a-1edfa20d715e None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:03.907342 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4c31d-6cc4-4dd0-b20a-1edfa20d715e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:03.909347 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51a5f3-f815-4b9b-8e0d-d0b71a2a7afe admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:03.914096 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ffdf0-aab2-4d57-a2c2-b11817c4af01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:03.915170 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ffdf0-aab2-4d57-a2c2-b11817c4af01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:03.915455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ffdf0-aab2-4d57-a2c2-b11817c4af01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:03.915734 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ffdf0-aab2-4d57-a2c2-b11817c4af01 admin admin] PATH_INFO: `/v3/projects/0502b9ef01e2460893cfc855d44db0c7/users/851454212d14438d80bf8e193fea268e/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:03.925410 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-72a4c31d-6cc4-4dd0-b20a-1edfa20d715e None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:03.926649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-72a4c31d-6cc4-4dd0-b20a-1edfa20d715e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:03.932140 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ffdf0-aab2-4d57-a2c2-b11817c4af01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0502b9ef01e2460893cfc855d44db0c7, user_id=851454212d14438d80bf8e193fea268e, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:03.934454 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 791/1594] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:03.935332 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ffdf0-aab2-4d57-a2c2-b11817c4af01 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:03.938406 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4c31d-6cc4-4dd0-b20a-1edfa20d715e None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:03.949345 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4c31d-6cc4-4dd0-b20a-1edfa20d715e None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:03.951316 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 804/1595] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:03 2022] PUT /identity/v3/projects/0502b9ef01e2460893cfc855d44db0c7/users/851454212d14438d80bf8e193fea268e/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:03.953063 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ffdf0-aab2-4d57-a2c2-b11817c4af01 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:03.970866 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:03.983464 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:04.008480 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3480434-1667-414d-8499-7e79dfe00064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:04.009154 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3480434-1667-414d-8499-7e79dfe00064 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:04.009426 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3480434-1667-414d-8499-7e79dfe00064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:04.009709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3480434-1667-414d-8499-7e79dfe00064 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:04.010628 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3480434-1667-414d-8499-7e79dfe00064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:04.012392 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3480434-1667-414d-8499-7e79dfe00064 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:04.016625 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ffdf0-aab2-4d57-a2c2-b11817c4af01 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:04.021725 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964dab15-35e5-4cef-8016-19c6abbf1113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:04.022465 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964dab15-35e5-4cef-8016-19c6abbf1113 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:04.022704 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964dab15-35e5-4cef-8016-19c6abbf1113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:04.022985 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964dab15-35e5-4cef-8016-19c6abbf1113 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:04.023726 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964dab15-35e5-4cef-8016-19c6abbf1113 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:04.025368 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964dab15-35e5-4cef-8016-19c6abbf1113 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:04.026748 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 792/1596] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:04.027523 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3480434-1667-414d-8499-7e79dfe00064 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:04.036327 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 805/1597] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:04.037251 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964dab15-35e5-4cef-8016-19c6abbf1113 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:04.041700 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3480434-1667-414d-8499-7e79dfe00064 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:04.046574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76f9cc-fa01-489c-a877-f455c523ec5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:04.047236 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76f9cc-fa01-489c-a877-f455c523ec5d admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:04.047490 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964dab15-35e5-4cef-8016-19c6abbf1113 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:04.047618 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76f9cc-fa01-489c-a877-f455c523ec5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:04.047794 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76f9cc-fa01-489c-a877-f455c523ec5d admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:04.048518 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76f9cc-fa01-489c-a877-f455c523ec5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:04.050435 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76f9cc-fa01-489c-a877-f455c523ec5d admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:04.052101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f3cf9-3400-498e-878b-96da89debadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:04.052777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f3cf9-3400-498e-878b-96da89debadc admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:04.053066 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f3cf9-3400-498e-878b-96da89debadc admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:04.053348 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f3cf9-3400-498e-878b-96da89debadc admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:04.054173 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14f3cf9-3400-498e-878b-96da89debadc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:04.055739 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14f3cf9-3400-498e-878b-96da89debadc admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:04.068955 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 806/1598] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:04.069762 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f3cf9-3400-498e-878b-96da89debadc admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:04.073377 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 793/1599] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:33:04 2022] POST /identity/v3/users => generated 307 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:04.074470 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76f9cc-fa01-489c-a877-f455c523ec5d admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:04.080009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f3cf9-3400-498e-878b-96da89debadc admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:04.084377 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76f9cc-fa01-489c-a877-f455c523ec5d admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:04.084690 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6390fcf-7453-4b5c-843a-8fdc5b8534e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:04.085569 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6390fcf-7453-4b5c-843a-8fdc5b8534e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:04.085861 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6390fcf-7453-4b5c-843a-8fdc5b8534e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:04.086145 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6390fcf-7453-4b5c-843a-8fdc5b8534e2 admin admin] PATH_INFO: `/v3/projects/dcc45b8bab17498cadda8d64b3a76780/users/687208d733174bffa2783e6857401f39/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:04.090083 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c95cc1-a78d-4698-a206-c18203e53323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:04.090749 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c95cc1-a78d-4698-a206-c18203e53323 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:04.091016 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c95cc1-a78d-4698-a206-c18203e53323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:04.091292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c95cc1-a78d-4698-a206-c18203e53323 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:04.092141 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c95cc1-a78d-4698-a206-c18203e53323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:04.093768 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c95cc1-a78d-4698-a206-c18203e53323 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:04.101647 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6390fcf-7453-4b5c-843a-8fdc5b8534e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcc45b8bab17498cadda8d64b3a76780, user_id=687208d733174bffa2783e6857401f39, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:04.104958 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6390fcf-7453-4b5c-843a-8fdc5b8534e2 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:04.107541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 794/1600] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:04.108475 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c95cc1-a78d-4698-a206-c18203e53323 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:04.123369 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 807/1601] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:04 2022] PUT /identity/v3/projects/dcc45b8bab17498cadda8d64b3a76780/users/687208d733174bffa2783e6857401f39/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:04.124837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f598d-9e50-42d6-af12-18b036cae0d6 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:04.125135 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f598d-9e50-42d6-af12-18b036cae0d6 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:04.125412 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f598d-9e50-42d6-af12-18b036cae0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:04.143271 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-e84f598d-9e50-42d6-af12-18b036cae0d6 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:04.144272 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e84f598d-9e50-42d6-af12-18b036cae0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:04.167767 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c95cc1-a78d-4698-a206-c18203e53323 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:04.210235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 808/1602] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:04.211534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f598d-9e50-42d6-af12-18b036cae0d6 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:04.221448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414d8bd-1f45-4fa4-9ebd-88b49a3f06a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:04.222117 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414d8bd-1f45-4fa4-9ebd-88b49a3f06a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:04.222407 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414d8bd-1f45-4fa4-9ebd-88b49a3f06a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:04.222690 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414d8bd-1f45-4fa4-9ebd-88b49a3f06a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:04.226240 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f598d-9e50-42d6-af12-18b036cae0d6 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:04.232495 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686bcd2-b6ca-428f-aafc-55e36ea0da8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:04.233174 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686bcd2-b6ca-428f-aafc-55e36ea0da8e admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:04.233455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686bcd2-b6ca-428f-aafc-55e36ea0da8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:04.233732 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686bcd2-b6ca-428f-aafc-55e36ea0da8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:04.234258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414d8bd-1f45-4fa4-9ebd-88b49a3f06a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:04.234447 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686bcd2-b6ca-428f-aafc-55e36ea0da8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:04.235787 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414d8bd-1f45-4fa4-9ebd-88b49a3f06a7 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:04.236336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686bcd2-b6ca-428f-aafc-55e36ea0da8e admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:04.246386 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 795/1603] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:04.247248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414d8bd-1f45-4fa4-9ebd-88b49a3f06a7 service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:04.248644 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 809/1604] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:04.251592 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686bcd2-b6ca-428f-aafc-55e36ea0da8e admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:04.257711 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414d8bd-1f45-4fa4-9ebd-88b49a3f06a7 service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:04.263090 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686bcd2-b6ca-428f-aafc-55e36ea0da8e admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:04.264616 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf2c05-79b6-44ac-a5fe-09cd741ef4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:04.265394 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf2c05-79b6-44ac-a5fe-09cd741ef4c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:04.265746 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf2c05-79b6-44ac-a5fe-09cd741ef4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:04.266090 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf2c05-79b6-44ac-a5fe-09cd741ef4c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:04.266884 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adf2c05-79b6-44ac-a5fe-09cd741ef4c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:04.268394 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b9106e-ece3-4c7c-ac01-4b4eb1c15f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:04.269378 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adf2c05-79b6-44ac-a5fe-09cd741ef4c2 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:04.269500 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9106e-ece3-4c7c-ac01-4b4eb1c15f2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:04.269717 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9106e-ece3-4c7c-ac01-4b4eb1c15f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:04.269997 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9106e-ece3-4c7c-ac01-4b4eb1c15f2c admin admin] PATH_INFO: `/v3/projects/da82f9c566664cb1a974cf0d1bf1b09d/users/ef222fe025534a9e942f8c5c4ed1e3e7/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:04.282048 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 796/1605] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:04.283161 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf2c05-79b6-44ac-a5fe-09cd741ef4c2 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:04.296687 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b9106e-ece3-4c7c-ac01-4b4eb1c15f2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=da82f9c566664cb1a974cf0d1bf1b09d, user_id=ef222fe025534a9e942f8c5c4ed1e3e7, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:04.297937 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b9106e-ece3-4c7c-ac01-4b4eb1c15f2c admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:04.299869 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf2c05-79b6-44ac-a5fe-09cd741ef4c2 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:04.307685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63acf203-db96-4402-9689-93a406b1826d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:04.309126 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63acf203-db96-4402-9689-93a406b1826d admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:04.309750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63acf203-db96-4402-9689-93a406b1826d admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:04.310407 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63acf203-db96-4402-9689-93a406b1826d admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:04.311862 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63acf203-db96-4402-9689-93a406b1826d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:04.315044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63acf203-db96-4402-9689-93a406b1826d admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:04.315338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 810/1606] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:04 2022] PUT /identity/v3/projects/da82f9c566664cb1a974cf0d1bf1b09d/users/ef222fe025534a9e942f8c5c4ed1e3e7/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:04.316325 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b9106e-ece3-4c7c-ac01-4b4eb1c15f2c admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:04.343912 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 797/1607] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:04.343912 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c2a4b-712e-48b7-8fc0-828395f58777 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:04.343912 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c2a4b-712e-48b7-8fc0-828395f58777 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:04.344704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c2a4b-712e-48b7-8fc0-828395f58777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:04.368608 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-a34c2a4b-712e-48b7-8fc0-828395f58777 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:04.370468 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-a34c2a4b-712e-48b7-8fc0-828395f58777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:04.388371 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:04.414099 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b9106e-ece3-4c7c-ac01-4b4eb1c15f2c admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:04.420797 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c057ec6-2869-4b49-a0af-d849b2323bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:04.422358 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c057ec6-2869-4b49-a0af-d849b2323bbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:04.422828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c057ec6-2869-4b49-a0af-d849b2323bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:04.423241 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c057ec6-2869-4b49-a0af-d849b2323bbb admin admin] PATH_INFO: `/v3/projects/736376830ed340b2aa62c455c134c4e2/users/771d3e22beff44f7bbc3103fd75a35b4/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:04.442495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:04.444075 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c057ec6-2869-4b49-a0af-d849b2323bbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=736376830ed340b2aa62c455c134c4e2, user_id=771d3e22beff44f7bbc3103fd75a35b4, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:04.446576 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c057ec6-2869-4b49-a0af-d849b2323bbb admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:04.471271 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 811/1608] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:04 2022] PUT /identity/v3/projects/736376830ed340b2aa62c455c134c4e2/users/771d3e22beff44f7bbc3103fd75a35b4/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:04.472465 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c057ec6-2869-4b49-a0af-d849b2323bbb admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:04.484116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 798/1609] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:04.484116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c2a4b-712e-48b7-8fc0-828395f58777 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:04.493118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c2a4b-712e-48b7-8fc0-828395f58777 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:04.555882 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c057ec6-2869-4b49-a0af-d849b2323bbb admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:04.559922 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebbcda1-bbb3-4937-a2af-6067a9992767 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:04.560699 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbcda1-bbb3-4937-a2af-6067a9992767 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:04.561104 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbcda1-bbb3-4937-a2af-6067a9992767 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:04.561398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbcda1-bbb3-4937-a2af-6067a9992767 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:04.584551 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0bf898-b4e2-438f-a4a2-d554983ab83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:04.585443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0bf898-b4e2-438f-a4a2-d554983ab83c admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:04.585733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0bf898-b4e2-438f-a4a2-d554983ab83c admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:04.586341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0bf898-b4e2-438f-a4a2-d554983ab83c admin admin] PATH_INFO: `/v3/projects/fd23dbbbe6d74bb897559fbab9c9b5f6/users/ecce945c45994e6faf78f6f5ae551c3b/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:04.613007 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0bf898-b4e2-438f-a4a2-d554983ab83c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd23dbbbe6d74bb897559fbab9c9b5f6, user_id=ecce945c45994e6faf78f6f5ae551c3b, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:04.617903 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0bf898-b4e2-438f-a4a2-d554983ab83c admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:04.640250 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 799/1610] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:04 2022] PUT /identity/v3/projects/fd23dbbbe6d74bb897559fbab9c9b5f6/users/ecce945c45994e6faf78f6f5ae551c3b/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:04.642049 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0bf898-b4e2-438f-a4a2-d554983ab83c admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:04.662296 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebbcda1-bbb3-4937-a2af-6067a9992767 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:04.664346 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebbcda1-bbb3-4937-a2af-6067a9992767 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:04.679755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 812/1611] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:04.681296 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976630ad-f352-440e-9660-5d879e283ef9 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:04.681687 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976630ad-f352-440e-9660-5d879e283ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:04.682292 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976630ad-f352-440e-9660-5d879e283ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:04.701840 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-976630ad-f352-440e-9660-5d879e283ef9 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:04.702897 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-976630ad-f352-440e-9660-5d879e283ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:04.758060 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0bf898-b4e2-438f-a4a2-d554983ab83c admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:04.765422 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b1e47-c31b-4588-a972-f73bbcb36a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:04.766521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b1e47-c31b-4588-a972-f73bbcb36a68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:04.766803 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b1e47-c31b-4588-a972-f73bbcb36a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:04.767077 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b1e47-c31b-4588-a972-f73bbcb36a68 admin admin] PATH_INFO: `/v3/projects/07449c42d3314784a550849550bf96bd/users/53d478ceb520434190dca479eff47ef7/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:04.789224 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b1e47-c31b-4588-a972-f73bbcb36a68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07449c42d3314784a550849550bf96bd, user_id=53d478ceb520434190dca479eff47ef7, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:04.792962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b1e47-c31b-4588-a972-f73bbcb36a68 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:04.810798 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 800/1612] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:04 2022] PUT /identity/v3/projects/07449c42d3314784a550849550bf96bd/users/53d478ceb520434190dca479eff47ef7/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:04.811797 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b1e47-c31b-4588-a972-f73bbcb36a68 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:04.834416 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 813/1613] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:04.842472 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa589731-c06b-4dea-97b6-40838ffa6e50 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:04.842812 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa589731-c06b-4dea-97b6-40838ffa6e50 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:04.843035 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa589731-c06b-4dea-97b6-40838ffa6e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:04.869019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-aa589731-c06b-4dea-97b6-40838ffa6e50 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:04.870982 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-aa589731-c06b-4dea-97b6-40838ffa6e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:04.884640 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:04.926299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:04.956933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b1e47-c31b-4588-a972-f73bbcb36a68 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:04.958716 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 814/1614] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:04.960602 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa589731-c06b-4dea-97b6-40838ffa6e50 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:04.965179 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ffdbfc-40c7-44a9-ab38-cc46fdb992ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:04.966526 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ffdbfc-40c7-44a9-ab38-cc46fdb992ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:04.967435 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ffdbfc-40c7-44a9-ab38-cc46fdb992ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:04.967815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ffdbfc-40c7-44a9-ab38-cc46fdb992ab admin admin] PATH_INFO: `/v3/projects/cb0d4466daea40fca1eabd5da38dfe95/users/35661998891c4ce783a4da6ddf0e6a45/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:04.989014 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ffdbfc-40c7-44a9-ab38-cc46fdb992ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb0d4466daea40fca1eabd5da38dfe95, user_id=35661998891c4ce783a4da6ddf0e6a45, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:04.993147 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ffdbfc-40c7-44a9-ab38-cc46fdb992ab admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:05.018660 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 801/1615] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:04 2022] PUT /identity/v3/projects/cb0d4466daea40fca1eabd5da38dfe95/users/35661998891c4ce783a4da6ddf0e6a45/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:05.023084 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f09f9-c285-4d5d-b45f-bc7300f46f5a None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:05.023084 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f09f9-c285-4d5d-b45f-bc7300f46f5a None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:05.023084 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f09f9-c285-4d5d-b45f-bc7300f46f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:05.038370 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa589731-c06b-4dea-97b6-40838ffa6e50 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:05.061487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-4e5f09f9-c285-4d5d-b45f-bc7300f46f5a None None] MFA Rules not processed for user `ecce945c45994e6faf78f6f5ae551c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:05.062803 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-4e5f09f9-c285-4d5d-b45f-bc7300f46f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:05.119233 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab036a61-5147-48ec-8c4c-d9ba487f5ed2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:05.120385 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab036a61-5147-48ec-8c4c-d9ba487f5ed2 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:05.120782 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab036a61-5147-48ec-8c4c-d9ba487f5ed2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:05.121161 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab036a61-5147-48ec-8c4c-d9ba487f5ed2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:05.198628 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab036a61-5147-48ec-8c4c-d9ba487f5ed2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:05.200412 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab036a61-5147-48ec-8c4c-d9ba487f5ed2 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:05.214686 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 815/1616] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:05.216240 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21e229-a8b2-43e8-ba7e-c9fc174ec927 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:05.216543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21e229-a8b2-43e8-ba7e-c9fc174ec927 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:05.216822 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21e229-a8b2-43e8-ba7e-c9fc174ec927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:05.260644 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-0c21e229-a8b2-43e8-ba7e-c9fc174ec927 None None] MFA Rules not processed for user `53d478ceb520434190dca479eff47ef7`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:05.262134 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-0c21e229-a8b2-43e8-ba7e-c9fc174ec927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:05.273683 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 802/1617] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:05 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:05.273683 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f09f9-c285-4d5d-b45f-bc7300f46f5a None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:05.286949 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:05.293722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f09f9-c285-4d5d-b45f-bc7300f46f5a None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:05.304763 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e53daa-f14e-4ab3-b95c-badf39b8ca88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:05.308574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e53daa-f14e-4ab3-b95c-badf39b8ca88 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:05.308927 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e53daa-f14e-4ab3-b95c-badf39b8ca88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:05.309178 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e53daa-f14e-4ab3-b95c-badf39b8ca88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:05.329936 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e53daa-f14e-4ab3-b95c-badf39b8ca88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:05.331994 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e53daa-f14e-4ab3-b95c-badf39b8ca88 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:05.349796 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 803/1618] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:05.350696 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e53daa-f14e-4ab3-b95c-badf39b8ca88 service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:05.364284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e53daa-f14e-4ab3-b95c-badf39b8ca88 service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:05.370996 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 816/1619] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:05 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:05.372245 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21e229-a8b2-43e8-ba7e-c9fc174ec927 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:05.376315 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4fcf3c-1e57-4124-a999-30a2a94bbd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:05.377126 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4fcf3c-1e57-4124-a999-30a2a94bbd3e admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:05.377894 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4fcf3c-1e57-4124-a999-30a2a94bbd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:05.380043 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4fcf3c-1e57-4124-a999-30a2a94bbd3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:05.381276 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4fcf3c-1e57-4124-a999-30a2a94bbd3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:05.383471 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4fcf3c-1e57-4124-a999-30a2a94bbd3e admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:05.384517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21e229-a8b2-43e8-ba7e-c9fc174ec927 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:05.392490 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b7884-76d6-46a1-b224-3da1c9a997b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:05.393186 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b7884-76d6-46a1-b224-3da1c9a997b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:05.393481 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b7884-76d6-46a1-b224-3da1c9a997b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:05.393785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b7884-76d6-46a1-b224-3da1c9a997b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:05.402374 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 804/1620] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:05.403948 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4fcf3c-1e57-4124-a999-30a2a94bbd3e admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:05.418779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4fcf3c-1e57-4124-a999-30a2a94bbd3e admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:05.422357 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b7884-76d6-46a1-b224-3da1c9a997b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:05.424020 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b7884-76d6-46a1-b224-3da1c9a997b2 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:05.426210 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261b519-1885-4e02-9a3a-f329bf1853fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:05.427073 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261b519-1885-4e02-9a3a-f329bf1853fe admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:05.427418 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261b519-1885-4e02-9a3a-f329bf1853fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:05.427927 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261b519-1885-4e02-9a3a-f329bf1853fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:05.428867 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261b519-1885-4e02-9a3a-f329bf1853fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:05.432327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261b519-1885-4e02-9a3a-f329bf1853fe admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:05.437908 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 817/1621] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:05.438762 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b7884-76d6-46a1-b224-3da1c9a997b2 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:05.506619 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 805/1622] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:33:05 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:05.508999 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261b519-1885-4e02-9a3a-f329bf1853fe admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:05.538537 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b7884-76d6-46a1-b224-3da1c9a997b2 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:05.656117 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261b519-1885-4e02-9a3a-f329bf1853fe admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:05.669694 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e20c5-bf39-4d8d-872e-b62857609cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:05.670975 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78938d24-6dea-4748-b11e-366d80bd5ae5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:05.671665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78938d24-6dea-4748-b11e-366d80bd5ae5 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:05.672789 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78938d24-6dea-4748-b11e-366d80bd5ae5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:05.673040 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78938d24-6dea-4748-b11e-366d80bd5ae5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:05.678796 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e20c5-bf39-4d8d-872e-b62857609cbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:05.679549 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e20c5-bf39-4d8d-872e-b62857609cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:05.680052 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e20c5-bf39-4d8d-872e-b62857609cbc admin admin] PATH_INFO: `/v3/projects/cb0d4466daea40fca1eabd5da38dfe95/users/35661998891c4ce783a4da6ddf0e6a45/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:05.687945 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450e20c5-bf39-4d8d-872e-b62857609cbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb0d4466daea40fca1eabd5da38dfe95, user_id=35661998891c4ce783a4da6ddf0e6a45, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:05.692311 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450e20c5-bf39-4d8d-872e-b62857609cbc admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:05.722584 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 806/1623] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:05 2022] PUT /identity/v3/projects/cb0d4466daea40fca1eabd5da38dfe95/users/35661998891c4ce783a4da6ddf0e6a45/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:05.725900 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a4427-0e86-48d5-8fce-c63844c3ccf9 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:05.726495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a4427-0e86-48d5-8fce-c63844c3ccf9 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:05.726945 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a4427-0e86-48d5-8fce-c63844c3ccf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:05.755404 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78938d24-6dea-4748-b11e-366d80bd5ae5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:05.757280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78938d24-6dea-4748-b11e-366d80bd5ae5 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:05.757933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:05.770367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 818/1624] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:05 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:05.771407 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78938d24-6dea-4748-b11e-366d80bd5ae5 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:05.773185 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-cb3a4427-0e86-48d5-8fce-c63844c3ccf9 None None] MFA Rules not processed for user `f2936fe7026046aa9c43f1b1b0cb1bfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:05.776271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-cb3a4427-0e86-48d5-8fce-c63844c3ccf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:05.784114 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78938d24-6dea-4748-b11e-366d80bd5ae5 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:05.790393 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:05.863896 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d17b2-d61e-497e-809e-c1006eb3d2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:05.865130 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d17b2-d61e-497e-809e-c1006eb3d2c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:05.865259 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d17b2-d61e-497e-809e-c1006eb3d2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:05.865577 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d17b2-d61e-497e-809e-c1006eb3d2c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:05.866642 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6d17b2-d61e-497e-809e-c1006eb3d2c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:05.868842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6d17b2-d61e-497e-809e-c1006eb3d2c7 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:05.898103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 819/1625] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:33:05 2022] POST /identity/v3/users => generated 289 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:05.899695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b88aa4-0d14-4f4f-aeb3-767f4f1b759a None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:05.900112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b88aa4-0d14-4f4f-aeb3-767f4f1b759a None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:05.901170 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b88aa4-0d14-4f4f-aeb3-767f4f1b759a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:05.918373 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 807/1626] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:05.922020 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3a4427-0e86-48d5-8fce-c63844c3ccf9 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:05.941258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3a4427-0e86-48d5-8fce-c63844c3ccf9 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:05.949251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf3924f-8725-4280-8bc6-82b3eff7df6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:05.951078 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3924f-8725-4280-8bc6-82b3eff7df6a admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:05.951464 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3924f-8725-4280-8bc6-82b3eff7df6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:05.951882 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3924f-8725-4280-8bc6-82b3eff7df6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:05.952967 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-71b88aa4-0d14-4f4f-aeb3-767f4f1b759a None None] MFA Rules not processed for user `35661998891c4ce783a4da6ddf0e6a45`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:05.953102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf3924f-8725-4280-8bc6-82b3eff7df6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:05.954962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf3924f-8725-4280-8bc6-82b3eff7df6a admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:05.955899 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-71b88aa4-0d14-4f4f-aeb3-767f4f1b759a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:05.972358 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 808/1627] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:05 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:05.972358 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e97a29-9a8c-4e07-b035-44c5dfe623df None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:05.972358 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e97a29-9a8c-4e07-b035-44c5dfe623df None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:05.972358 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e97a29-9a8c-4e07-b035-44c5dfe623df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:05.999826 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-d6e97a29-9a8c-4e07-b035-44c5dfe623df None None] MFA Rules not processed for user `f2936fe7026046aa9c43f1b1b0cb1bfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:06.001365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-d6e97a29-9a8c-4e07-b035-44c5dfe623df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:06.041819 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 809/1628] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:06.047702 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e97a29-9a8c-4e07-b035-44c5dfe623df None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:06.072694 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e97a29-9a8c-4e07-b035-44c5dfe623df None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:06.087970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e4607-39b9-4ab6-91c0-d257195cda1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:06.100038 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e4607-39b9-4ab6-91c0-d257195cda1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:06.100384 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e4607-39b9-4ab6-91c0-d257195cda1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:06.107270 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e4607-39b9-4ab6-91c0-d257195cda1c admin admin] PATH_INFO: `/v3/projects/14767d6407b645efade02ea237a97ede/users/bb74ea2e2d674481ad47e1cccdc57112/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:06.156739 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e4607-39b9-4ab6-91c0-d257195cda1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=14767d6407b645efade02ea237a97ede, user_id=bb74ea2e2d674481ad47e1cccdc57112, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:06.159455 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e4607-39b9-4ab6-91c0-d257195cda1c admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:06.169872 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 820/1629] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:05 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:06.172497 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9025c5-90b9-4b1c-a192-4d894ae69030 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:06.172805 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9025c5-90b9-4b1c-a192-4d894ae69030 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:06.173093 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9025c5-90b9-4b1c-a192-4d894ae69030 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:06.174657 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 821/1630] 10.222.0.16 () {54 vars in 961 bytes} [Mon May 23 12:33:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:33:06.185412 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85479f3a-36fe-42d6-b77a-1a811bdb9f73 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:06.186933 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85479f3a-36fe-42d6-b77a-1a811bdb9f73 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:06.187419 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85479f3a-36fe-42d6-b77a-1a811bdb9f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:06.203387 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 810/1631] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:06 2022] PUT /identity/v3/projects/14767d6407b645efade02ea237a97ede/users/bb74ea2e2d674481ad47e1cccdc57112/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:06.206239 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e4607-39b9-4ab6-91c0-d257195cda1c admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:06.264789 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-85479f3a-36fe-42d6-b77a-1a811bdb9f73 None None] MFA Rules not processed for user `b83535b443a84275a160cd84cb3ad5a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:06.270928 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-85479f3a-36fe-42d6-b77a-1a811bdb9f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:06.394458 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e4607-39b9-4ab6-91c0-d257195cda1c admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:06.399532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c513b59-06fb-4d81-b849-d53903c5f356 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:06.400377 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c513b59-06fb-4d81-b849-d53903c5f356 service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:06.401249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c513b59-06fb-4d81-b849-d53903c5f356 service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:06.401627 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c513b59-06fb-4d81-b849-d53903c5f356 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:06.494699 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 822/1632] 10.222.0.16 () {58 vars in 1058 bytes} [Mon May 23 12:33:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:06.496586 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85479f3a-36fe-42d6-b77a-1a811bdb9f73 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:06.509489 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c513b59-06fb-4d81-b849-d53903c5f356 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:06.511240 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c513b59-06fb-4d81-b849-d53903c5f356 service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:06.521713 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:06.523479 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 811/1633] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:06 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:06.526258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c513b59-06fb-4d81-b849-d53903c5f356 service cinder] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:06.539311 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c513b59-06fb-4d81-b849-d53903c5f356 service cinder] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:06.546701 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1a6ee-7ad7-46d7-8b10-95affc406b2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:06.547537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1a6ee-7ad7-46d7-8b10-95affc406b2c service nova] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:06.547932 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1a6ee-7ad7-46d7-8b10-95affc406b2c service nova] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:06.548343 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1a6ee-7ad7-46d7-8b10-95affc406b2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:06.558897 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:06.647777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85479f3a-36fe-42d6-b77a-1a811bdb9f73 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:06.656662 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264e798-4427-4403-be47-779b6ba07d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:06.657515 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264e798-4427-4403-be47-779b6ba07d44 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:06.658642 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264e798-4427-4403-be47-779b6ba07d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:06.659752 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264e798-4427-4403-be47-779b6ba07d44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:06.660773 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3264e798-4427-4403-be47-779b6ba07d44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:06.662883 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3264e798-4427-4403-be47-779b6ba07d44 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:06.687531 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 823/1634] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:06.688320 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264e798-4427-4403-be47-779b6ba07d44 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:06.688818 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1a6ee-7ad7-46d7-8b10-95affc406b2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:06.690658 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1a6ee-7ad7-46d7-8b10-95affc406b2c service nova] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:06.702439 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 812/1635] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:06.703367 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1a6ee-7ad7-46d7-8b10-95affc406b2c service nova] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:06.715441 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1a6ee-7ad7-46d7-8b10-95affc406b2c service nova] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:06.720649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63ce73-4144-4d11-bd06-9a666275d35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:06.721752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63ce73-4144-4d11-bd06-9a666275d35e admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:06.722144 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63ce73-4144-4d11-bd06-9a666275d35e admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:06.722554 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63ce73-4144-4d11-bd06-9a666275d35e admin admin] PATH_INFO: `/v3/projects/14767d6407b645efade02ea237a97ede/users/bb74ea2e2d674481ad47e1cccdc57112/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:06.724866 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b63ce73-4144-4d11-bd06-9a666275d35e admin admin] RBAC: Authorizing `identity:create_grant(project_id=14767d6407b645efade02ea237a97ede, user_id=bb74ea2e2d674481ad47e1cccdc57112, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:06.727908 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b63ce73-4144-4d11-bd06-9a666275d35e admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:06.738696 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264e798-4427-4403-be47-779b6ba07d44 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:06.740876 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 813/1636] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:06 2022] PUT /identity/v3/projects/14767d6407b645efade02ea237a97ede/users/bb74ea2e2d674481ad47e1cccdc57112/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:06.753768 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50f88d-1d65-446f-afd3-754900881645 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:06.755901 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50f88d-1d65-446f-afd3-754900881645 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:06.755901 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50f88d-1d65-446f-afd3-754900881645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:06.788859 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-cb50f88d-1d65-446f-afd3-754900881645 None None] MFA Rules not processed for user `bb74ea2e2d674481ad47e1cccdc57112`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:06.790583 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-cb50f88d-1d65-446f-afd3-754900881645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:06.892476 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 814/1637] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:06 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:33:06.907419 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0bd010-1989-4a1b-ac00-280617f28ca6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:06.909628 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0bd010-1989-4a1b-ac00-280617f28ca6 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:06.910170 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0bd010-1989-4a1b-ac00-280617f28ca6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:06.910531 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0bd010-1989-4a1b-ac00-280617f28ca6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:07.007781 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50f88d-1d65-446f-afd3-754900881645 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:07.024162 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50f88d-1d65-446f-afd3-754900881645 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:07.031937 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c23ecd-a7e6-466e-8ccd-6847533fa7f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:07.032864 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c23ecd-a7e6-466e-8ccd-6847533fa7f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:07.033790 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c23ecd-a7e6-466e-8ccd-6847533fa7f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:07.034182 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c23ecd-a7e6-466e-8ccd-6847533fa7f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:07.051737 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c23ecd-a7e6-466e-8ccd-6847533fa7f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:07.053883 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c23ecd-a7e6-466e-8ccd-6847533fa7f7 service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:07.070729 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 815/1638] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:07 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:07.141065 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0bd010-1989-4a1b-ac00-280617f28ca6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:07.147558 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0bd010-1989-4a1b-ac00-280617f28ca6 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:07.180562 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 824/1639] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:06 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:07.352975 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c23ecd-a7e6-466e-8ccd-6847533fa7f7 service cinder] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:07.363042 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c23ecd-a7e6-466e-8ccd-6847533fa7f7 service cinder] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:07.376727 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140d026-e552-4238-b6a1-465cfcbe81b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:07.379588 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140d026-e552-4238-b6a1-465cfcbe81b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:07.380044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140d026-e552-4238-b6a1-465cfcbe81b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:07.380817 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140d026-e552-4238-b6a1-465cfcbe81b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:07.450692 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe7674-fa6c-4b47-96a7-da4cb06f7b13 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:07.451187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe7674-fa6c-4b47-96a7-da4cb06f7b13 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:07.451488 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe7674-fa6c-4b47-96a7-da4cb06f7b13 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:07.453867 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 825/1640] 10.222.0.16 () {54 vars in 912 bytes} [Mon May 23 12:33:07 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:33:07.459114 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdac8f-7ba1-4ef2-a9e1-b2ad42437c09 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:07.459405 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdac8f-7ba1-4ef2-a9e1-b2ad42437c09 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:07.459924 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdac8f-7ba1-4ef2-a9e1-b2ad42437c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:07.588996 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-86bdac8f-7ba1-4ef2-a9e1-b2ad42437c09 None None] MFA Rules not processed for user `b83535b443a84275a160cd84cb3ad5a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:07.595417 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-86bdac8f-7ba1-4ef2-a9e1-b2ad42437c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:07.673159 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140d026-e552-4238-b6a1-465cfcbe81b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:07.678197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140d026-e552-4238-b6a1-465cfcbe81b4 service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:07.715341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 816/1641] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:07 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:07.792378 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:07.837495 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 826/1642] 10.222.0.16 () {58 vars in 1009 bytes} [Mon May 23 12:33:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:07.848218 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140d026-e552-4238-b6a1-465cfcbe81b4 service cinder] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:07.851291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-b140d026-e552-4238-b6a1-465cfcbe81b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:07.869377 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:08.022898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140d026-e552-4238-b6a1-465cfcbe81b4 service cinder] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:08.034716 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ed516-385f-42a4-a1ed-0077887192b3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2483a351f5e943ad8c872a98a4717b0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:08.034716 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ed516-385f-42a4-a1ed-0077887192b3 service glance] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:08.037609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ed516-385f-42a4-a1ed-0077887192b3 service glance] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:08.037724 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ed516-385f-42a4-a1ed-0077887192b3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:08.098985 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ed516-385f-42a4-a1ed-0077887192b3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:08.100698 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ed516-385f-42a4-a1ed-0077887192b3 service glance] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:08.171558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 817/1643] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:08.642445 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e4681-16a9-4a5a-8a99-dcae3c4049cc None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:08.643319 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e4681-16a9-4a5a-8a99-dcae3c4049cc None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:08.643477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e4681-16a9-4a5a-8a99-dcae3c4049cc None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:08.646361 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 827/1644] 10.222.0.16 () {54 vars in 961 bytes} [Mon May 23 12:33:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:33:08.660099 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e31209-483d-4e46-ae84-250a6fa215e3 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:08.660875 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e31209-483d-4e46-ae84-250a6fa215e3 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:08.661498 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e31209-483d-4e46-ae84-250a6fa215e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:08.692928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-18e31209-483d-4e46-ae84-250a6fa215e3 None None] MFA Rules not processed for user `b83535b443a84275a160cd84cb3ad5a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:08.694542 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-18e31209-483d-4e46-ae84-250a6fa215e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:08.711807 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 818/1645] 10.222.0.16 () {58 vars in 1058 bytes} [Mon May 23 12:33:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:08.719941 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc4981-72c2-4ca0-9368-5899057b3209 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:08.720307 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc4981-72c2-4ca0-9368-5899057b3209 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:08.720713 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc4981-72c2-4ca0-9368-5899057b3209 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:08.722614 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 828/1646] 10.222.0.16 () {54 vars in 912 bytes} [Mon May 23 12:33:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:33:08.749185 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f34b7-7edd-4405-a5a1-f2a6b80164f4 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:08.749478 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f34b7-7edd-4405-a5a1-f2a6b80164f4 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:08.749759 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f34b7-7edd-4405-a5a1-f2a6b80164f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:08.799329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-234f34b7-7edd-4405-a5a1-f2a6b80164f4 None None] MFA Rules not processed for user `b83535b443a84275a160cd84cb3ad5a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:08.801558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-234f34b7-7edd-4405-a5a1-f2a6b80164f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:08.818988 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 819/1647] 10.222.0.16 () {58 vars in 1009 bytes} [Mon May 23 12:33:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:08.833650 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc4981-72c2-4ca0-9368-5899057b3209 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:08.862244 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc4981-72c2-4ca0-9368-5899057b3209 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:08.867300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b1f8e-2120-40cc-ba57-987b25abb663 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:08.868209 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b1f8e-2120-40cc-ba57-987b25abb663 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:08.869041 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b1f8e-2120-40cc-ba57-987b25abb663 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:08.870880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b1f8e-2120-40cc-ba57-987b25abb663 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:08.884648 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b7fe8-bdc3-493c-8ebb-b9744a13c681 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:08.885408 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b7fe8-bdc3-493c-8ebb-b9744a13c681 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:08.886270 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b7fe8-bdc3-493c-8ebb-b9744a13c681 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:08.889362 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 820/1648] 10.222.0.16 () {54 vars in 912 bytes} [Mon May 23 12:33:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:33:08.892193 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811b1f8e-2120-40cc-ba57-987b25abb663 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:08.893665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811b1f8e-2120-40cc-ba57-987b25abb663 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:08.895061 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc48fa5-b164-4a68-b6a5-e2fb9e333db4 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:08.895680 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc48fa5-b164-4a68-b6a5-e2fb9e333db4 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:08.896209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc48fa5-b164-4a68-b6a5-e2fb9e333db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:08.929916 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 829/1649] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:09.011563 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-1cc48fa5-b164-4a68-b6a5-e2fb9e333db4 None None] MFA Rules not processed for user `b6677be3ddf647e2b971bb3ba2c34a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:09.013872 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-1cc48fa5-b164-4a68-b6a5-e2fb9e333db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:09.159261 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 821/1650] 10.222.0.16 () {58 vars in 1009 bytes} [Mon May 23 12:33:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:33:09.177623 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b1f8e-2120-40cc-ba57-987b25abb663 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:09.196316 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b1f8e-2120-40cc-ba57-987b25abb663 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:09.206699 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f2e17-64ac-4476-8bd2-736a398822ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:09.209586 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f2e17-64ac-4476-8bd2-736a398822ad service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:09.211421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f2e17-64ac-4476-8bd2-736a398822ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:09.211946 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f2e17-64ac-4476-8bd2-736a398822ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:09.235746 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856f2e17-64ac-4476-8bd2-736a398822ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:09.242271 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856f2e17-64ac-4476-8bd2-736a398822ad service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:09.267591 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 830/1651] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:33:10.482428 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e71204-ca6b-4b50-ad7a-5fb2637c1f34 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:10.483459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e71204-ca6b-4b50-ad7a-5fb2637c1f34 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:10.483940 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e71204-ca6b-4b50-ad7a-5fb2637c1f34 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:10.486174 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 822/1652] 10.222.0.16 () {54 vars in 960 bytes} [Mon May 23 12:33:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 12:33:10.498781 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7326e-eeae-4a87-8822-4efe92f83789 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:10.499307 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7326e-eeae-4a87-8822-4efe92f83789 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:10.499729 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7326e-eeae-4a87-8822-4efe92f83789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:10.562474 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-f4d7326e-eeae-4a87-8822-4efe92f83789 None None] MFA Rules not processed for user `ebc0f137e0d24dd2a460689bc17a1b98`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:10.563837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-f4d7326e-eeae-4a87-8822-4efe92f83789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:10.718700 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 831/1653] 10.222.0.16 () {58 vars in 1057 bytes} [Mon May 23 12:33:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:10.720802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e71204-ca6b-4b50-ad7a-5fb2637c1f34 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:10.753451 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e71204-ca6b-4b50-ad7a-5fb2637c1f34 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:10.761984 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c37cea-ce5d-48f4-9f4b-139725af8d3f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc0f137e0d24dd2a460689bc17a1b98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:10.763376 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c37cea-ce5d-48f4-9f4b-139725af8d3f service placement] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:10.763851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c37cea-ce5d-48f4-9f4b-139725af8d3f service placement] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:10.764526 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c37cea-ce5d-48f4-9f4b-139725af8d3f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:10.768354 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-64c37cea-ce5d-48f4-9f4b-139725af8d3f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:10.810520 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c37cea-ce5d-48f4-9f4b-139725af8d3f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:10.820186 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c37cea-ce5d-48f4-9f4b-139725af8d3f service placement] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:10.865581 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 823/1654] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:13.398289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7326e-eeae-4a87-8822-4efe92f83789 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:13.604745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7326e-eeae-4a87-8822-4efe92f83789 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:13.628519 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad433eb4-19da-4d5e-acb9-75d640983a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:13.629681 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad433eb4-19da-4d5e-acb9-75d640983a28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:13.630103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad433eb4-19da-4d5e-acb9-75d640983a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:13.630540 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad433eb4-19da-4d5e-acb9-75d640983a28 admin admin] PATH_INFO: `/v3/users/04f89ee7b66247d28c44ffbad8b119f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:13.632070 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad433eb4-19da-4d5e-acb9-75d640983a28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04f89ee7b66247d28c44ffbad8b119f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:13.634169 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad433eb4-19da-4d5e-acb9-75d640983a28 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:13.777384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ad433eb4-19da-4d5e-acb9-75d640983a28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04f89ee7b66247d28c44ffbad8b119f8'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:13.806054 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ad433eb4-19da-4d5e-acb9-75d640983a28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04f89ee7b66247d28c44ffbad8b119f8'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:13.812284 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:13.837017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ad433eb4-19da-4d5e-acb9-75d640983a28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04f89ee7b66247d28c44ffbad8b119f8'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:13.838460 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ad433eb4-19da-4d5e-acb9-75d640983a28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04f89ee7b66247d28c44ffbad8b119f8'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:13.839797 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ad433eb4-19da-4d5e-acb9-75d640983a28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04f89ee7b66247d28c44ffbad8b119f8'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:13.889602 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 832/1655] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:33:13 2022] DELETE /identity/v3/users/04f89ee7b66247d28c44ffbad8b119f8 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:13.894221 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41e137-55a7-449e-8069-b3bf8326c5e0 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:13.894811 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41e137-55a7-449e-8069-b3bf8326c5e0 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:13.894811 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41e137-55a7-449e-8069-b3bf8326c5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:13.929489 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-ef41e137-55a7-449e-8069-b3bf8326c5e0 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:13.932443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-ef41e137-55a7-449e-8069-b3bf8326c5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:14.075290 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 824/1656] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:14.098377 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad433eb4-19da-4d5e-acb9-75d640983a28 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:14.100586 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ad433eb4-19da-4d5e-acb9-75d640983a28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:14.274171 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad433eb4-19da-4d5e-acb9-75d640983a28 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:14.294540 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c581c-dd5a-4c2f-9ba8-234c144658af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:14.299468 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c581c-dd5a-4c2f-9ba8-234c144658af service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:14.299895 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c581c-dd5a-4c2f-9ba8-234c144658af service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:14.300796 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c581c-dd5a-4c2f-9ba8-234c144658af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:14.329982 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4c581c-dd5a-4c2f-9ba8-234c144658af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:14.331605 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4c581c-dd5a-4c2f-9ba8-234c144658af service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:14.375801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 833/1657] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:14.910036 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41e137-55a7-449e-8069-b3bf8326c5e0 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:14.912055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-ef41e137-55a7-449e-8069-b3bf8326c5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:14.946505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41e137-55a7-449e-8069-b3bf8326c5e0 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:14.962947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5374f997-7739-4286-ade9-f72516f91a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:14.966113 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374f997-7739-4286-ade9-f72516f91a36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:14.966612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374f997-7739-4286-ade9-f72516f91a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:14.967132 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374f997-7739-4286-ade9-f72516f91a36 admin admin] PATH_INFO: `/v3/projects/abbd5b0e8cf742cfa1429996026462d4` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:14.970411 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5374f997-7739-4286-ade9-f72516f91a36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abbd5b0e8cf742cfa1429996026462d4)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:14.978067 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5374f997-7739-4286-ade9-f72516f91a36 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:15.035753 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:15.188862 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-5374f997-7739-4286-ade9-f72516f91a36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abbd5b0e8cf742cfa1429996026462d4', 'request_id': 'req-5374f997-7739-4286-ade9-f72516f91a36'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:15.208372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-5374f997-7739-4286-ade9-f72516f91a36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abbd5b0e8cf742cfa1429996026462d4', 'request_id': 'req-5374f997-7739-4286-ade9-f72516f91a36'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:15.218693 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-5374f997-7739-4286-ade9-f72516f91a36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abbd5b0e8cf742cfa1429996026462d4', 'request_id': 'req-5374f997-7739-4286-ade9-f72516f91a36'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:15.233967 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-5374f997-7739-4286-ade9-f72516f91a36 admin admin] The token cache is being invalidate because project abbd5b0e8cf742cfa1429996026462d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:33:15.234491 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-5374f997-7739-4286-ade9-f72516f91a36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:15.235385 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-5374f997-7739-4286-ade9-f72516f91a36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:15.248790 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 825/1658] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:33:14 2022] DELETE /identity/v3/projects/abbd5b0e8cf742cfa1429996026462d4 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:33:15.252833 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c581c-dd5a-4c2f-9ba8-234c144658af service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:15.275089 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c581c-dd5a-4c2f-9ba8-234c144658af service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:15.394892 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3159c1-c4ae-429b-b3a9-d0ee7a296684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:15.396097 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3159c1-c4ae-429b-b3a9-d0ee7a296684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:15.396686 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3159c1-c4ae-429b-b3a9-d0ee7a296684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:15.397247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3159c1-c4ae-429b-b3a9-d0ee7a296684 admin admin] PATH_INFO: `/v3/users/35661998891c4ce783a4da6ddf0e6a45` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:15.398892 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3159c1-c4ae-429b-b3a9-d0ee7a296684 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35661998891c4ce783a4da6ddf0e6a45)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:15.402099 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3159c1-c4ae-429b-b3a9-d0ee7a296684 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:15.498098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ba3159c1-c4ae-429b-b3a9-d0ee7a296684 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35661998891c4ce783a4da6ddf0e6a45'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:15.513897 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ba3159c1-c4ae-429b-b3a9-d0ee7a296684 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35661998891c4ce783a4da6ddf0e6a45'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:15.525286 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ba3159c1-c4ae-429b-b3a9-d0ee7a296684 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35661998891c4ce783a4da6ddf0e6a45'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:15.526751 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ba3159c1-c4ae-429b-b3a9-d0ee7a296684 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35661998891c4ce783a4da6ddf0e6a45'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:15.527673 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ba3159c1-c4ae-429b-b3a9-d0ee7a296684 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35661998891c4ce783a4da6ddf0e6a45'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:15.548640 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 834/1659] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:33:15 2022] DELETE /identity/v3/users/35661998891c4ce783a4da6ddf0e6a45 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:15.979836 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5374f997-7739-4286-ade9-f72516f91a36 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:15.983661 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-5374f997-7739-4286-ade9-f72516f91a36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:16.028856 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5374f997-7739-4286-ade9-f72516f91a36 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:16.217492 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868c6b0-60f9-4d62-985f-02879436520a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:16.219214 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868c6b0-60f9-4d62-985f-02879436520a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:16.227490 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868c6b0-60f9-4d62-985f-02879436520a admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:16.228239 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868c6b0-60f9-4d62-985f-02879436520a admin admin] PATH_INFO: `/v3/projects/cb0d4466daea40fca1eabd5da38dfe95` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:16.231679 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868c6b0-60f9-4d62-985f-02879436520a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb0d4466daea40fca1eabd5da38dfe95)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:16.246601 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868c6b0-60f9-4d62-985f-02879436520a admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:16.369887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2868c6b0-60f9-4d62-985f-02879436520a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb0d4466daea40fca1eabd5da38dfe95', 'request_id': 'req-2868c6b0-60f9-4d62-985f-02879436520a'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:16.388595 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2868c6b0-60f9-4d62-985f-02879436520a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb0d4466daea40fca1eabd5da38dfe95', 'request_id': 'req-2868c6b0-60f9-4d62-985f-02879436520a'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:16.398757 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2868c6b0-60f9-4d62-985f-02879436520a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb0d4466daea40fca1eabd5da38dfe95', 'request_id': 'req-2868c6b0-60f9-4d62-985f-02879436520a'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:16.463343 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2868c6b0-60f9-4d62-985f-02879436520a admin admin] The token cache is being invalidate because project cb0d4466daea40fca1eabd5da38dfe95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:33:16.463628 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2868c6b0-60f9-4d62-985f-02879436520a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:16.469412 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2868c6b0-60f9-4d62-985f-02879436520a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:16.476941 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 826/1660] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:33:15 2022] DELETE /identity/v3/projects/cb0d4466daea40fca1eabd5da38dfe95 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:16.482885 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e8824-f349-44bd-a7c5-ef4cf72dcc6b None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:16.483397 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e8824-f349-44bd-a7c5-ef4cf72dcc6b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:16.483532 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e8824-f349-44bd-a7c5-ef4cf72dcc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:16.532953 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-339e8824-f349-44bd-a7c5-ef4cf72dcc6b None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:16.534620 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-339e8824-f349-44bd-a7c5-ef4cf72dcc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:16.540208 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:16.692487 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 835/1661] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:16.693871 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f43112-e999-49b3-8413-d1b9cecfa61e None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:16.694372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f43112-e999-49b3-8413-d1b9cecfa61e None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:16.694803 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f43112-e999-49b3-8413-d1b9cecfa61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:16.724895 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-a0f43112-e999-49b3-8413-d1b9cecfa61e None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:16.726217 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-a0f43112-e999-49b3-8413-d1b9cecfa61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:16.746421 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 827/1662] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:16.751520 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e8824-f349-44bd-a7c5-ef4cf72dcc6b None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:16.767571 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e8824-f349-44bd-a7c5-ef4cf72dcc6b None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:16.789053 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3240c414-8e15-491e-b0ed-57c499291569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:16.789927 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3240c414-8e15-491e-b0ed-57c499291569 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:16.793489 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3240c414-8e15-491e-b0ed-57c499291569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:16.793909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3240c414-8e15-491e-b0ed-57c499291569 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:16.796056 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3240c414-8e15-491e-b0ed-57c499291569 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:33:16.799334 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3240c414-8e15-491e-b0ed-57c499291569 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:16.802430 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3240c414-8e15-491e-b0ed-57c499291569 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:16.850664 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 836/1663] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:33:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:33:16.856469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f43112-e999-49b3-8413-d1b9cecfa61e None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:16.876619 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f43112-e999-49b3-8413-d1b9cecfa61e None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:16.894867 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4753214b-b2f7-4b45-a6e2-690e178557fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:16.897407 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753214b-b2f7-4b45-a6e2-690e178557fc admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:16.897711 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753214b-b2f7-4b45-a6e2-690e178557fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:16.899697 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4753214b-b2f7-4b45-a6e2-690e178557fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:16.901769 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4753214b-b2f7-4b45-a6e2-690e178557fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:16.907584 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4753214b-b2f7-4b45-a6e2-690e178557fc admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:16.959595 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:16.975492 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 828/1664] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:33:16 2022] POST /identity/v3/projects => generated 407 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:16.976756 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3240c414-8e15-491e-b0ed-57c499291569 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:16.991060 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3240c414-8e15-491e-b0ed-57c499291569 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:17.038915 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578a498-5780-47ae-b663-bfc55627b3c2 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:17.039438 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578a498-5780-47ae-b663-bfc55627b3c2 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:17.039554 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578a498-5780-47ae-b663-bfc55627b3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:17.099977 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-7578a498-5780-47ae-b663-bfc55627b3c2 None None] MFA Rules not processed for user `733f8296ec3d4795a1140c1e1cb6eb95`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:17.101439 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-7578a498-5780-47ae-b663-bfc55627b3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:17.186367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c35035a-df15-4466-8443-93bb8b1ef53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:17.187340 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35035a-df15-4466-8443-93bb8b1ef53d admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:17.188630 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35035a-df15-4466-8443-93bb8b1ef53d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:17.189625 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35035a-df15-4466-8443-93bb8b1ef53d admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:17.191581 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c35035a-df15-4466-8443-93bb8b1ef53d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:17.196124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c35035a-df15-4466-8443-93bb8b1ef53d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:17.268423 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 837/1665] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:33:16 2022] POST /identity/v3/users => generated 304 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:17.269758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c35035a-df15-4466-8443-93bb8b1ef53d admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:17.282921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 829/1666] 10.222.0.16 () {58 vars in 1009 bytes} [Mon May 23 12:33:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:17.296589 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c35035a-df15-4466-8443-93bb8b1ef53d admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:17.310220 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ead96c-f94c-4dd4-9f27-73c6a9779c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:17.310927 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ead96c-f94c-4dd4-9f27-73c6a9779c71 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:17.311272 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ead96c-f94c-4dd4-9f27-73c6a9779c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:17.311610 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ead96c-f94c-4dd4-9f27-73c6a9779c71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:17.312483 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ead96c-f94c-4dd4-9f27-73c6a9779c71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:17.317369 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ead96c-f94c-4dd4-9f27-73c6a9779c71 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:17.355159 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578a498-5780-47ae-b663-bfc55627b3c2 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:17.355396 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 838/1667] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:17.394454 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578a498-5780-47ae-b663-bfc55627b3c2 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:17.432514 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1efc1a7-45bc-47a2-8606-473c72177cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:17.433719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1efc1a7-45bc-47a2-8606-473c72177cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:17.434222 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1efc1a7-45bc-47a2-8606-473c72177cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:17.434506 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1efc1a7-45bc-47a2-8606-473c72177cb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:17.435268 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1efc1a7-45bc-47a2-8606-473c72177cb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:17.436960 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1efc1a7-45bc-47a2-8606-473c72177cb8 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:17.454935 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ead96c-f94c-4dd4-9f27-73c6a9779c71 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:17.462290 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 830/1668] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:17.467979 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ead96c-f94c-4dd4-9f27-73c6a9779c71 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:17.477377 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e0a66c-24f2-4e12-b095-350498d39a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:17.478371 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0a66c-24f2-4e12-b095-350498d39a6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:17.478660 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0a66c-24f2-4e12-b095-350498d39a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:17.478941 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0a66c-24f2-4e12-b095-350498d39a6d admin admin] PATH_INFO: `/v3/projects/02c94f61c8fa4f16a11818e663776ac5/users/00a6d045a92d48799bd91d84bedc383d/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:17.528219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1efc1a7-45bc-47a2-8606-473c72177cb8 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:17.530225 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-f1efc1a7-45bc-47a2-8606-473c72177cb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:17.533947 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e0a66c-24f2-4e12-b095-350498d39a6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=02c94f61c8fa4f16a11818e663776ac5, user_id=00a6d045a92d48799bd91d84bedc383d, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:17.537123 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e0a66c-24f2-4e12-b095-350498d39a6d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:17.566267 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 839/1669] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:17 2022] PUT /identity/v3/projects/02c94f61c8fa4f16a11818e663776ac5/users/00a6d045a92d48799bd91d84bedc383d/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:17.582848 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846625a0-733f-411b-b474-f2883aa8ff53 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:17.583107 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846625a0-733f-411b-b474-f2883aa8ff53 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:17.593744 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846625a0-733f-411b-b474-f2883aa8ff53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:17.714323 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-846625a0-733f-411b-b474-f2883aa8ff53 None None] MFA Rules not processed for user `00a6d045a92d48799bd91d84bedc383d`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:17.715581 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-846625a0-733f-411b-b474-f2883aa8ff53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:17.795237 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1efc1a7-45bc-47a2-8606-473c72177cb8 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:17.849543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:17.927724 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 840/1670] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:17 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:17.931150 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846625a0-733f-411b-b474-f2883aa8ff53 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:18.013353 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8446cd2-4bcc-495d-bacf-42ab1a6836af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71fb9a79ac494635804b93d4a0048205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:18.014295 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8446cd2-4bcc-495d-bacf-42ab1a6836af service swift] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:18.022649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8446cd2-4bcc-495d-bacf-42ab1a6836af service swift] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:18.022649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8446cd2-4bcc-495d-bacf-42ab1a6836af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:18.186215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:18.195190 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846625a0-733f-411b-b474-f2883aa8ff53 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:18.210101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a98b8-1cd7-4d97-a277-8bc516242400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:18.213570 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a98b8-1cd7-4d97-a277-8bc516242400 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:18.215697 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a98b8-1cd7-4d97-a277-8bc516242400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:18.215998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a98b8-1cd7-4d97-a277-8bc516242400 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:18.219433 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a98b8-1cd7-4d97-a277-8bc516242400 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:18.231874 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a98b8-1cd7-4d97-a277-8bc516242400 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:18.289086 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8446cd2-4bcc-495d-bacf-42ab1a6836af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:18.296500 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8446cd2-4bcc-495d-bacf-42ab1a6836af service swift] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:18.319975 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 841/1671] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:33:17 2022] POST /identity/v3/projects => generated 409 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:18.326213 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 831/1672] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 12:33:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 801 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 23 12:33:18.337801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a98b8-1cd7-4d97-a277-8bc516242400 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:18.657916 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a98b8-1cd7-4d97-a277-8bc516242400 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:18.686083 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f3160-5a97-4a2e-b3e4-4fa8a47c0075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:18.690004 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f3160-5a97-4a2e-b3e4-4fa8a47c0075 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:18.690300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f3160-5a97-4a2e-b3e4-4fa8a47c0075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:18.695434 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f3160-5a97-4a2e-b3e4-4fa8a47c0075 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:18.696535 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f3160-5a97-4a2e-b3e4-4fa8a47c0075 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:18.709086 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f3160-5a97-4a2e-b3e4-4fa8a47c0075 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:18.799897 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 842/1673] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:33:18 2022] POST /identity/v3/users => generated 311 bytes in 474 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:18.802510 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8446cd2-4bcc-495d-bacf-42ab1a6836af service swift] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:18.821591 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8446cd2-4bcc-495d-bacf-42ab1a6836af service swift] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:18.838665 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916f7ae-c314-4a46-ba44-ccada64c0a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:18.839601 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916f7ae-c314-4a46-ba44-ccada64c0a0f admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:18.840318 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916f7ae-c314-4a46-ba44-ccada64c0a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:18.840731 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8916f7ae-c314-4a46-ba44-ccada64c0a0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:18.841605 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8916f7ae-c314-4a46-ba44-ccada64c0a0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:18.857397 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8916f7ae-c314-4a46-ba44-ccada64c0a0f admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:18.898711 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 832/1674] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:18.904376 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f3160-5a97-4a2e-b3e4-4fa8a47c0075 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:18.930452 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f3160-5a97-4a2e-b3e4-4fa8a47c0075 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:18.953941 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47113ab-d99d-430b-8fb7-69bea5f2bce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:18.958571 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47113ab-d99d-430b-8fb7-69bea5f2bce6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:18.960086 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47113ab-d99d-430b-8fb7-69bea5f2bce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:18.961922 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47113ab-d99d-430b-8fb7-69bea5f2bce6 admin admin] PATH_INFO: `/v3/projects/b1619875a23e4d548255a0fcf87cd307/users/bf53ea54ee20453f9ed1995b1503309d/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:19.046916 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47113ab-d99d-430b-8fb7-69bea5f2bce6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1619875a23e4d548255a0fcf87cd307, user_id=bf53ea54ee20453f9ed1995b1503309d, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:19.050245 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47113ab-d99d-430b-8fb7-69bea5f2bce6 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:19.096659 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 843/1675] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:18 2022] PUT /identity/v3/projects/b1619875a23e4d548255a0fcf87cd307/users/bf53ea54ee20453f9ed1995b1503309d/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:19.101188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916f7ae-c314-4a46-ba44-ccada64c0a0f admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:19.124967 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8916f7ae-c314-4a46-ba44-ccada64c0a0f admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:19.406699 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47113ab-d99d-430b-8fb7-69bea5f2bce6 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:19.409128 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e47113ab-d99d-430b-8fb7-69bea5f2bce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:19.453178 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247f1c4-e3f1-4456-a3a0-e0aa3be154c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:19.454802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247f1c4-e3f1-4456-a3a0-e0aa3be154c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:19.454802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247f1c4-e3f1-4456-a3a0-e0aa3be154c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:19.454802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247f1c4-e3f1-4456-a3a0-e0aa3be154c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:19.456968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247f1c4-e3f1-4456-a3a0-e0aa3be154c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:19.458832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247f1c4-e3f1-4456-a3a0-e0aa3be154c6 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:19.493352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 833/1676] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:19.498894 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247f1c4-e3f1-4456-a3a0-e0aa3be154c6 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:19.536367 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247f1c4-e3f1-4456-a3a0-e0aa3be154c6 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:19.537944 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:19.561690 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e75b06d-9ed5-4021-bd56-af09186b9c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:19.562562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75b06d-9ed5-4021-bd56-af09186b9c90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:19.562948 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75b06d-9ed5-4021-bd56-af09186b9c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:19.563196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e75b06d-9ed5-4021-bd56-af09186b9c90 admin admin] PATH_INFO: `/v3/projects/b1619875a23e4d548255a0fcf87cd307/users/bf53ea54ee20453f9ed1995b1503309d/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:19.579158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e75b06d-9ed5-4021-bd56-af09186b9c90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1619875a23e4d548255a0fcf87cd307, user_id=bf53ea54ee20453f9ed1995b1503309d, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:19.590293 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e75b06d-9ed5-4021-bd56-af09186b9c90 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:19.632900 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47113ab-d99d-430b-8fb7-69bea5f2bce6 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:19.656697 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 834/1677] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:19 2022] PUT /identity/v3/projects/b1619875a23e4d548255a0fcf87cd307/users/bf53ea54ee20453f9ed1995b1503309d/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:33:19.665731 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d079a0-344a-4943-993c-57c9e714ea8a None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:19.666446 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d079a0-344a-4943-993c-57c9e714ea8a None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:19.666758 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d079a0-344a-4943-993c-57c9e714ea8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:19.783800 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-04d079a0-344a-4943-993c-57c9e714ea8a None None] MFA Rules not processed for user `bf53ea54ee20453f9ed1995b1503309d`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:19.791614 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-04d079a0-344a-4943-993c-57c9e714ea8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:19.821344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f90929-92b9-4efb-94d5-4b3434776aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:19.822205 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f90929-92b9-4efb-94d5-4b3434776aeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:19.822619 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f90929-92b9-4efb-94d5-4b3434776aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:19.823110 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f90929-92b9-4efb-94d5-4b3434776aeb admin admin] PATH_INFO: `/v3/users/53d478ceb520434190dca479eff47ef7` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:19.831209 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f90929-92b9-4efb-94d5-4b3434776aeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=53d478ceb520434190dca479eff47ef7)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:19.833224 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f90929-92b9-4efb-94d5-4b3434776aeb admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:19.937466 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-96f90929-92b9-4efb-94d5-4b3434776aeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53d478ceb520434190dca479eff47ef7'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:19.964619 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-96f90929-92b9-4efb-94d5-4b3434776aeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53d478ceb520434190dca479eff47ef7'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:19.980724 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-96f90929-92b9-4efb-94d5-4b3434776aeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53d478ceb520434190dca479eff47ef7'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:19.982268 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-96f90929-92b9-4efb-94d5-4b3434776aeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53d478ceb520434190dca479eff47ef7'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:19.983169 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-96f90929-92b9-4efb-94d5-4b3434776aeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53d478ceb520434190dca479eff47ef7'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:20.018861 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 835/1678] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:19 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:20.032754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d079a0-344a-4943-993c-57c9e714ea8a None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:20.035065 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-04d079a0-344a-4943-993c-57c9e714ea8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:20.060593 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 844/1679] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:33:19 2022] DELETE /identity/v3/users/53d478ceb520434190dca479eff47ef7 => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:20.067726 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e414aec-dc28-4753-8ddc-968332b24159 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:20.071270 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e414aec-dc28-4753-8ddc-968332b24159 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:20.072294 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e414aec-dc28-4753-8ddc-968332b24159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:20.123236 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-5e414aec-dc28-4753-8ddc-968332b24159 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:20.127223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5e414aec-dc28-4753-8ddc-968332b24159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:20.139007 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:20.187348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d079a0-344a-4943-993c-57c9e714ea8a None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:20.301420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93204317-67c1-410e-81d4-4b43bf272286 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:20.302108 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93204317-67c1-410e-81d4-4b43bf272286 service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:20.302439 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93204317-67c1-410e-81d4-4b43bf272286 service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:20.302730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93204317-67c1-410e-81d4-4b43bf272286 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:20.443622 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93204317-67c1-410e-81d4-4b43bf272286 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:20.445184 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93204317-67c1-410e-81d4-4b43bf272286 service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:20.474404 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 836/1680] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:20 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:33:20.668959 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:20.697030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 845/1681] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 632 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:20.840936 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93204317-67c1-410e-81d4-4b43bf272286 service cinder] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:20.842973 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-93204317-67c1-410e-81d4-4b43bf272286 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:20.887716 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93204317-67c1-410e-81d4-4b43bf272286 service cinder] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:20.894240 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db926d77-d265-40c5-b773-323301834fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:20.895138 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db926d77-d265-40c5-b773-323301834fc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:20.895420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db926d77-d265-40c5-b773-323301834fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:20.895751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db926d77-d265-40c5-b773-323301834fc2 admin admin] PATH_INFO: `/v3/users/ecce945c45994e6faf78f6f5ae551c3b` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:20.897099 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db926d77-d265-40c5-b773-323301834fc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecce945c45994e6faf78f6f5ae551c3b)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:20.898151 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: WARNING py.warnings [None req-db926d77-d265-40c5-b773-323301834fc2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 12:33:20.898151 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: warnings.warn(msg) May 23 12:33:20.898151 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: May 23 12:33:20.899855 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db926d77-d265-40c5-b773-323301834fc2 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:20.948603 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e414aec-dc28-4753-8ddc-968332b24159 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:20.950450 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5e414aec-dc28-4753-8ddc-968332b24159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:20.991041 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e414aec-dc28-4753-8ddc-968332b24159 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:20.996246 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5acda0b-b5f5-4098-9d2b-82e9234998e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:20.996969 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5acda0b-b5f5-4098-9d2b-82e9234998e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:20.997251 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5acda0b-b5f5-4098-9d2b-82e9234998e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:20.997535 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5acda0b-b5f5-4098-9d2b-82e9234998e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:21.004692 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d5acda0b-b5f5-4098-9d2b-82e9234998e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:21.066936 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-db926d77-d265-40c5-b773-323301834fc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecce945c45994e6faf78f6f5ae551c3b'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:21.068516 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-db926d77-d265-40c5-b773-323301834fc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecce945c45994e6faf78f6f5ae551c3b'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:21.103124 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-db926d77-d265-40c5-b773-323301834fc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecce945c45994e6faf78f6f5ae551c3b'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:21.157707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-db926d77-d265-40c5-b773-323301834fc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecce945c45994e6faf78f6f5ae551c3b'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:21.173057 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-db926d77-d265-40c5-b773-323301834fc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecce945c45994e6faf78f6f5ae551c3b'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:21.190643 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5acda0b-b5f5-4098-9d2b-82e9234998e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:21.192332 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5acda0b-b5f5-4098-9d2b-82e9234998e9 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:21.194936 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 837/1682] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:33:20 2022] DELETE /identity/v3/users/ecce945c45994e6faf78f6f5ae551c3b => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:21.201337 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa20927-fd72-46b2-97cf-581c46119ef6 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:21.203430 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa20927-fd72-46b2-97cf-581c46119ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:21.203778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa20927-fd72-46b2-97cf-581c46119ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:21.237419 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 846/1683] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:20 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:33:21.256642 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5acda0b-b5f5-4098-9d2b-82e9234998e9 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:21.266053 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d5acda0b-b5f5-4098-9d2b-82e9234998e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:21.281376 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-afa20927-fd72-46b2-97cf-581c46119ef6 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:21.282661 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-afa20927-fd72-46b2-97cf-581c46119ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:21.323523 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:21.392628 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5acda0b-b5f5-4098-9d2b-82e9234998e9 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:21.467291 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e156d-6f29-4d8e-b377-51cfd9d9a95c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:21.468149 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e156d-6f29-4d8e-b377-51cfd9d9a95c service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:21.468479 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e156d-6f29-4d8e-b377-51cfd9d9a95c service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:21.468889 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e156d-6f29-4d8e-b377-51cfd9d9a95c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:21.472209 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a02e156d-6f29-4d8e-b377-51cfd9d9a95c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:21.519774 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 838/1684] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:21.579581 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e156d-6f29-4d8e-b377-51cfd9d9a95c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:21.581670 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e156d-6f29-4d8e-b377-51cfd9d9a95c service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:21.595197 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 847/1685] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:21.674192 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa20927-fd72-46b2-97cf-581c46119ef6 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:21.675799 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-afa20927-fd72-46b2-97cf-581c46119ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:21.712576 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa20927-fd72-46b2-97cf-581c46119ef6 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:21.731572 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e545f6-1d6c-4cbd-b95e-52ae8f12d237 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:21.732441 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e545f6-1d6c-4cbd-b95e-52ae8f12d237 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:21.732739 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e545f6-1d6c-4cbd-b95e-52ae8f12d237 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:21.733064 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e545f6-1d6c-4cbd-b95e-52ae8f12d237 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:21.775442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e545f6-1d6c-4cbd-b95e-52ae8f12d237 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:21.776879 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e545f6-1d6c-4cbd-b95e-52ae8f12d237 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:21.808113 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 839/1686] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:33:22.673293 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a507e1-9cfe-40fc-9023-ea838a0be734 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:22.674192 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a507e1-9cfe-40fc-9023-ea838a0be734 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:22.674641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a507e1-9cfe-40fc-9023-ea838a0be734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:22.686502 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:22.717337 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-f1a507e1-9cfe-40fc-9023-ea838a0be734 None None] MFA Rules not processed for user `b6677be3ddf647e2b971bb3ba2c34a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:22.719444 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-f1a507e1-9cfe-40fc-9023-ea838a0be734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:22.735383 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 848/1687] 10.222.0.15 () {58 vars in 1015 bytes} [Mon May 23 12:33:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:22.811638 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e545f6-1d6c-4cbd-b95e-52ae8f12d237 service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:22.813251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-e8e545f6-1d6c-4cbd-b95e-52ae8f12d237 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:22.839212 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e545f6-1d6c-4cbd-b95e-52ae8f12d237 service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:22.845016 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce24ca12-f6b4-4574-a61e-491de30953fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:22.846136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24ca12-f6b4-4574-a61e-491de30953fd service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:22.846682 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24ca12-f6b4-4574-a61e-491de30953fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:22.847074 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24ca12-f6b4-4574-a61e-491de30953fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:22.872723 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce24ca12-f6b4-4574-a61e-491de30953fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:22.874317 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce24ca12-f6b4-4574-a61e-491de30953fd service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:22.900658 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 840/1688] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:23.137249 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a507e1-9cfe-40fc-9023-ea838a0be734 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:23.138650 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-f1a507e1-9cfe-40fc-9023-ea838a0be734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:23.165530 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a507e1-9cfe-40fc-9023-ea838a0be734 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:23.176430 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ca947-32b6-4da1-bdc3-bcde5210d1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:23.177111 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ca947-32b6-4da1-bdc3-bcde5210d1a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:23.177824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ca947-32b6-4da1-bdc3-bcde5210d1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:23.178114 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ca947-32b6-4da1-bdc3-bcde5210d1a6 admin admin] PATH_INFO: `/v3/projects/07449c42d3314784a550849550bf96bd` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:23.179333 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154ca947-32b6-4da1-bdc3-bcde5210d1a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07449c42d3314784a550849550bf96bd)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:23.188289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: WARNING py.warnings [None req-154ca947-32b6-4da1-bdc3-bcde5210d1a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 12:33:23.188289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: warnings.warn(msg) May 23 12:33:23.188289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: May 23 12:33:23.189602 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154ca947-32b6-4da1-bdc3-bcde5210d1a6 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:23.365035 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-154ca947-32b6-4da1-bdc3-bcde5210d1a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07449c42d3314784a550849550bf96bd', 'request_id': 'req-154ca947-32b6-4da1-bdc3-bcde5210d1a6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:23.381153 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-154ca947-32b6-4da1-bdc3-bcde5210d1a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07449c42d3314784a550849550bf96bd', 'request_id': 'req-154ca947-32b6-4da1-bdc3-bcde5210d1a6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:23.438042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-154ca947-32b6-4da1-bdc3-bcde5210d1a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07449c42d3314784a550849550bf96bd', 'request_id': 'req-154ca947-32b6-4da1-bdc3-bcde5210d1a6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:23.491604 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-154ca947-32b6-4da1-bdc3-bcde5210d1a6 admin admin] The token cache is being invalidate because project 07449c42d3314784a550849550bf96bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:33:23.492720 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-154ca947-32b6-4da1-bdc3-bcde5210d1a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:23.498511 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-154ca947-32b6-4da1-bdc3-bcde5210d1a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:23.510749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 849/1689] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:33:23 2022] DELETE /identity/v3/projects/07449c42d3314784a550849550bf96bd => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:23.516956 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a690d2-f58d-403e-8c68-0ea34f1436c5 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:23.517530 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a690d2-f58d-403e-8c68-0ea34f1436c5 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:23.518338 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a690d2-f58d-403e-8c68-0ea34f1436c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:23.604609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-97a690d2-f58d-403e-8c68-0ea34f1436c5 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:23.619523 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-97a690d2-f58d-403e-8c68-0ea34f1436c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:23.947228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 841/1690] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 433 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:23.957938 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87fe82c-f83f-44cc-a8a7-a377a405ba0b None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:23.958463 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87fe82c-f83f-44cc-a8a7-a377a405ba0b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:23.958550 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87fe82c-f83f-44cc-a8a7-a377a405ba0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:24.102008 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-d87fe82c-f83f-44cc-a8a7-a377a405ba0b None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:24.113777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d87fe82c-f83f-44cc-a8a7-a377a405ba0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:24.167961 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 850/1691] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:24.172857 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a690d2-f58d-403e-8c68-0ea34f1436c5 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:24.196586 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a690d2-f58d-403e-8c68-0ea34f1436c5 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:24.217307 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43cbb0-055c-4a20-8196-3a474975625b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:24.217307 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43cbb0-055c-4a20-8196-3a474975625b admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:24.217307 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43cbb0-055c-4a20-8196-3a474975625b admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:24.217307 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43cbb0-055c-4a20-8196-3a474975625b admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:24.217777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d43cbb0-055c-4a20-8196-3a474975625b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:33:24.217777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d43cbb0-055c-4a20-8196-3a474975625b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:24.217777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d43cbb0-055c-4a20-8196-3a474975625b admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:24.246679 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 842/1692] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:33:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:33:24.248552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87fe82c-f83f-44cc-a8a7-a377a405ba0b None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:24.280826 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87fe82c-f83f-44cc-a8a7-a377a405ba0b None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:24.299637 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa5ab7-8b4c-4ecb-91fb-6c12f8494fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:24.300369 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa5ab7-8b4c-4ecb-91fb-6c12f8494fc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:24.300672 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa5ab7-8b4c-4ecb-91fb-6c12f8494fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:24.300934 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa5ab7-8b4c-4ecb-91fb-6c12f8494fc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:24.301665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aa5ab7-8b4c-4ecb-91fb-6c12f8494fc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:24.303740 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aa5ab7-8b4c-4ecb-91fb-6c12f8494fc4 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:24.372212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 851/1693] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:33:24 2022] POST /identity/v3/projects => generated 363 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:33:24.378483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43cbb0-055c-4a20-8196-3a474975625b admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:24.414417 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43cbb0-055c-4a20-8196-3a474975625b admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:24.513804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef8071-f440-4bc7-9afd-d6eca7b92e64 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:24.513804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef8071-f440-4bc7-9afd-d6eca7b92e64 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:24.513804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef8071-f440-4bc7-9afd-d6eca7b92e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:24.581752 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-d9ef8071-f440-4bc7-9afd-d6eca7b92e64 None None] MFA Rules not processed for user `851454212d14438d80bf8e193fea268e`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:24.583013 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d9ef8071-f440-4bc7-9afd-d6eca7b92e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:24.705569 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097c58d-8bfb-43c2-bcd6-61e2974fa85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:24.706754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097c58d-8bfb-43c2-bcd6-61e2974fa85c admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:24.707744 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097c58d-8bfb-43c2-bcd6-61e2974fa85c admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:24.707744 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097c58d-8bfb-43c2-bcd6-61e2974fa85c admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:24.713988 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6097c58d-8bfb-43c2-bcd6-61e2974fa85c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:24.730566 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6097c58d-8bfb-43c2-bcd6-61e2974fa85c admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:24.885024 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 843/1694] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:33:24 2022] POST /identity/v3/users => generated 282 bytes in 511 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:24.892610 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097c58d-8bfb-43c2-bcd6-61e2974fa85c admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:24.904766 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:24.911516 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 852/1695] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:24.930002 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef8071-f440-4bc7-9afd-d6eca7b92e64 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:24.939229 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d9ef8071-f440-4bc7-9afd-d6eca7b92e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:24.940607 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097c58d-8bfb-43c2-bcd6-61e2974fa85c admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:24.959892 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67367a60-af17-4b2d-9ac3-56291b002ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:24.960942 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67367a60-af17-4b2d-9ac3-56291b002ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:24.961238 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67367a60-af17-4b2d-9ac3-56291b002ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:24.961514 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67367a60-af17-4b2d-9ac3-56291b002ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:24.962272 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67367a60-af17-4b2d-9ac3-56291b002ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:24.964986 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67367a60-af17-4b2d-9ac3-56291b002ce2 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:25.031467 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 844/1696] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:24 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:25.038275 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67367a60-af17-4b2d-9ac3-56291b002ce2 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:25.084486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67367a60-af17-4b2d-9ac3-56291b002ce2 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:25.129993 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:25.130699 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256d6db-53ff-47e2-8d39-a08c4f56d50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:25.132076 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256d6db-53ff-47e2-8d39-a08c4f56d50b admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:25.132643 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256d6db-53ff-47e2-8d39-a08c4f56d50b admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:25.133152 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256d6db-53ff-47e2-8d39-a08c4f56d50b admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:25.134506 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4256d6db-53ff-47e2-8d39-a08c4f56d50b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:25.148596 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4256d6db-53ff-47e2-8d39-a08c4f56d50b admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:25.204447 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 845/1697] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:25.204447 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256d6db-53ff-47e2-8d39-a08c4f56d50b admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:25.228505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256d6db-53ff-47e2-8d39-a08c4f56d50b admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:25.240263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b0636-bf34-4171-a686-6bca40c1c070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:25.243421 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b0636-bf34-4171-a686-6bca40c1c070 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:25.245168 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b0636-bf34-4171-a686-6bca40c1c070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:25.245716 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b0636-bf34-4171-a686-6bca40c1c070 admin admin] PATH_INFO: `/v3/projects/eb8f810a63bb403ba2f47529aba432a1/users/60fd535764604783ae128519bf0963e4/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:25.261818 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef8071-f440-4bc7-9afd-d6eca7b92e64 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:25.281746 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f2bee-5046-491e-82b2-8a0b347f9f74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:25.282572 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f2bee-5046-491e-82b2-8a0b347f9f74 service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:25.282709 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f2bee-5046-491e-82b2-8a0b347f9f74 service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:25.282983 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f2bee-5046-491e-82b2-8a0b347f9f74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:25.310330 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f2bee-5046-491e-82b2-8a0b347f9f74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:25.312034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f2bee-5046-491e-82b2-8a0b347f9f74 service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:25.314575 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b0636-bf34-4171-a686-6bca40c1c070 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb8f810a63bb403ba2f47529aba432a1, user_id=60fd535764604783ae128519bf0963e4, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:25.319742 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240b0636-bf34-4171-a686-6bca40c1c070 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:25.377159 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 853/1698] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:33:25.385516 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 846/1699] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:25 2022] PUT /identity/v3/projects/eb8f810a63bb403ba2f47529aba432a1/users/60fd535764604783ae128519bf0963e4/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:33:25.396749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1569db-0cb6-4011-b4e3-80339131366f None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:25.397104 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1569db-0cb6-4011-b4e3-80339131366f None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:25.397314 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1569db-0cb6-4011-b4e3-80339131366f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:25.535338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-ca1569db-0cb6-4011-b4e3-80339131366f None None] MFA Rules not processed for user `60fd535764604783ae128519bf0963e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:25.549164 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ca1569db-0cb6-4011-b4e3-80339131366f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:25.710627 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 854/1700] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:25 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:25.710944 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b0636-bf34-4171-a686-6bca40c1c070 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:26.008803 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b0636-bf34-4171-a686-6bca40c1c070 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:26.046147 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc2bed-a2b6-4163-b69c-94ad527dc98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:26.047328 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc2bed-a2b6-4163-b69c-94ad527dc98f admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:26.047699 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc2bed-a2b6-4163-b69c-94ad527dc98f admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:26.048153 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc2bed-a2b6-4163-b69c-94ad527dc98f admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:26.049371 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc2bed-a2b6-4163-b69c-94ad527dc98f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:26.083073 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc2bed-a2b6-4163-b69c-94ad527dc98f admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:26.157896 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 847/1701] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:33:25 2022] POST /identity/v3/projects => generated 363 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:26.162076 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1569db-0cb6-4011-b4e3-80339131366f None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:26.189492 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1569db-0cb6-4011-b4e3-80339131366f None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:26.338921 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730735af-3348-452a-914f-bd9ae8e5df5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:26.340270 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730735af-3348-452a-914f-bd9ae8e5df5d admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:26.340793 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730735af-3348-452a-914f-bd9ae8e5df5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:26.341283 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730735af-3348-452a-914f-bd9ae8e5df5d admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:26.342237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730735af-3348-452a-914f-bd9ae8e5df5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:26.344726 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730735af-3348-452a-914f-bd9ae8e5df5d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:26.390367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 855/1702] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:33:26 2022] POST /identity/v3/users => generated 288 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:26.394177 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc2bed-a2b6-4163-b69c-94ad527dc98f admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:26.408528 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc2bed-a2b6-4163-b69c-94ad527dc98f admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:26.440846 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8c941-cd05-4258-a4fd-c658d394c59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:26.443248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8c941-cd05-4258-a4fd-c658d394c59c admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:26.443862 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8c941-cd05-4258-a4fd-c658d394c59c admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:26.444986 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8c941-cd05-4258-a4fd-c658d394c59c admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:26.446145 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed8c941-cd05-4258-a4fd-c658d394c59c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:26.463006 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed8c941-cd05-4258-a4fd-c658d394c59c admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:26.515812 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 848/1703] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:26.519399 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730735af-3348-452a-914f-bd9ae8e5df5d admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:26.536772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730735af-3348-452a-914f-bd9ae8e5df5d admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:26.545618 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409621b-1f7f-4a02-b2d6-f441392ce349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:26.546982 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409621b-1f7f-4a02-b2d6-f441392ce349 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:26.547501 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409621b-1f7f-4a02-b2d6-f441392ce349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:26.549471 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409621b-1f7f-4a02-b2d6-f441392ce349 admin admin] PATH_INFO: `/v3/projects/9bce65ee673645868cf2cc29e0d81351/users/bf7bb760aa03484c859645de28832f72/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:26.603303 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5409621b-1f7f-4a02-b2d6-f441392ce349 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bce65ee673645868cf2cc29e0d81351, user_id=bf7bb760aa03484c859645de28832f72, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:26.618232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5409621b-1f7f-4a02-b2d6-f441392ce349 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:26.660958 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 856/1704] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:26 2022] PUT /identity/v3/projects/9bce65ee673645868cf2cc29e0d81351/users/bf7bb760aa03484c859645de28832f72/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:26.663010 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8c941-cd05-4258-a4fd-c658d394c59c admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:26.678571 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8c941-cd05-4258-a4fd-c658d394c59c admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:26.869552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409621b-1f7f-4a02-b2d6-f441392ce349 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:26.871498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5409621b-1f7f-4a02-b2d6-f441392ce349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:26.978477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:27.023198 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5caa16-60a9-44bb-b236-4ef18a606840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:27.023198 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5caa16-60a9-44bb-b236-4ef18a606840 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:27.023198 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5caa16-60a9-44bb-b236-4ef18a606840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:27.023198 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5caa16-60a9-44bb-b236-4ef18a606840 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:27.023936 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5caa16-60a9-44bb-b236-4ef18a606840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:27.025361 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5caa16-60a9-44bb-b236-4ef18a606840 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:27.096783 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 849/1705] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:27.097767 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5caa16-60a9-44bb-b236-4ef18a606840 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:27.158860 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5caa16-60a9-44bb-b236-4ef18a606840 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:27.188892 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5bd6a9-d80a-434f-98eb-c000cc816926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:27.189830 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5bd6a9-d80a-434f-98eb-c000cc816926 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:27.190121 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5bd6a9-d80a-434f-98eb-c000cc816926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:27.190448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5bd6a9-d80a-434f-98eb-c000cc816926 admin admin] PATH_INFO: `/v3/projects/9bce65ee673645868cf2cc29e0d81351/users/bf7bb760aa03484c859645de28832f72/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:27.193239 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409621b-1f7f-4a02-b2d6-f441392ce349 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:27.201095 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5bd6a9-d80a-434f-98eb-c000cc816926 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bce65ee673645868cf2cc29e0d81351, user_id=bf7bb760aa03484c859645de28832f72, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:27.203887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5bd6a9-d80a-434f-98eb-c000cc816926 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:27.221906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da69ad-c123-463a-9587-ab5873af07fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:27.222825 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da69ad-c123-463a-9587-ab5873af07fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:27.223115 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da69ad-c123-463a-9587-ab5873af07fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:27.223404 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da69ad-c123-463a-9587-ab5873af07fc admin admin] PATH_INFO: `/v3/projects/fd23dbbbe6d74bb897559fbab9c9b5f6` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:27.230042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da69ad-c123-463a-9587-ab5873af07fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd23dbbbe6d74bb897559fbab9c9b5f6)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:27.232030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da69ad-c123-463a-9587-ab5873af07fc admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:27.267511 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 850/1706] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:27 2022] PUT /identity/v3/projects/9bce65ee673645868cf2cc29e0d81351/users/bf7bb760aa03484c859645de28832f72/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:27.279718 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b06864-6650-4c8d-bbaf-9bd61529b437 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:27.280074 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b06864-6650-4c8d-bbaf-9bd61529b437 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:27.288302 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b06864-6650-4c8d-bbaf-9bd61529b437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:27.463510 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-d2b06864-6650-4c8d-bbaf-9bd61529b437 None None] MFA Rules not processed for user `bf7bb760aa03484c859645de28832f72`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:27.472448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-d2b06864-6650-4c8d-bbaf-9bd61529b437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:27.514325 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:27.559815 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-12da69ad-c123-463a-9587-ab5873af07fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd23dbbbe6d74bb897559fbab9c9b5f6', 'request_id': 'req-12da69ad-c123-463a-9587-ab5873af07fc'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:27.596687 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-12da69ad-c123-463a-9587-ab5873af07fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd23dbbbe6d74bb897559fbab9c9b5f6', 'request_id': 'req-12da69ad-c123-463a-9587-ab5873af07fc'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:27.608119 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-12da69ad-c123-463a-9587-ab5873af07fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd23dbbbe6d74bb897559fbab9c9b5f6', 'request_id': 'req-12da69ad-c123-463a-9587-ab5873af07fc'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:27.652834 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-12da69ad-c123-463a-9587-ab5873af07fc admin admin] The token cache is being invalidate because project fd23dbbbe6d74bb897559fbab9c9b5f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:33:27.653141 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-12da69ad-c123-463a-9587-ab5873af07fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:27.657941 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-12da69ad-c123-463a-9587-ab5873af07fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:27.661776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 857/1707] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:33:26 2022] DELETE /identity/v3/projects/fd23dbbbe6d74bb897559fbab9c9b5f6 => generated 0 bytes in 793 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:33:27.667636 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da69ad-c123-463a-9587-ab5873af07fc admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:27.675167 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-12da69ad-c123-463a-9587-ab5873af07fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:27.874212 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 851/1708] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:27 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 591 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:27.875568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b06864-6650-4c8d-bbaf-9bd61529b437 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:27.929477 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-d2b06864-6650-4c8d-bbaf-9bd61529b437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:28.011678 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da69ad-c123-463a-9587-ab5873af07fc admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:28.061261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc646d59-2614-446e-ace0-541c0b0c1f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:28.062187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc646d59-2614-446e-ace0-541c0b0c1f05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:28.062463 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc646d59-2614-446e-ace0-541c0b0c1f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:28.062749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc646d59-2614-446e-ace0-541c0b0c1f05 admin admin] PATH_INFO: `/v3/users/bb74ea2e2d674481ad47e1cccdc57112` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:28.065908 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc646d59-2614-446e-ace0-541c0b0c1f05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb74ea2e2d674481ad47e1cccdc57112)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:28.075839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc646d59-2614-446e-ace0-541c0b0c1f05 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:28.417397 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b06864-6650-4c8d-bbaf-9bd61529b437 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:28.436249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426dd72-7ec0-4c87-95f3-4120834e75ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:28.436945 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426dd72-7ec0-4c87-95f3-4120834e75ff service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:28.437240 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426dd72-7ec0-4c87-95f3-4120834e75ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:28.437524 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426dd72-7ec0-4c87-95f3-4120834e75ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:28.469153 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-dc646d59-2614-446e-ace0-541c0b0c1f05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb74ea2e2d674481ad47e1cccdc57112'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:28.490288 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:28.499216 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426dd72-7ec0-4c87-95f3-4120834e75ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:28.508594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426dd72-7ec0-4c87-95f3-4120834e75ff service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:28.542212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-dc646d59-2614-446e-ace0-541c0b0c1f05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb74ea2e2d674481ad47e1cccdc57112'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:28.558854 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 852/1709] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:27 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 684 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:28.574610 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-dc646d59-2614-446e-ace0-541c0b0c1f05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb74ea2e2d674481ad47e1cccdc57112'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:28.576540 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-dc646d59-2614-446e-ace0-541c0b0c1f05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb74ea2e2d674481ad47e1cccdc57112'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:28.577299 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-dc646d59-2614-446e-ace0-541c0b0c1f05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb74ea2e2d674481ad47e1cccdc57112'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:28.629868 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 858/1710] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:33:27 2022] DELETE /identity/v3/users/bb74ea2e2d674481ad47e1cccdc57112 => generated 0 bytes in 963 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:28.993113 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd86a5-39bf-45d3-a399-56d88697f38e None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:28.993415 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd86a5-39bf-45d3-a399-56d88697f38e None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:28.993784 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd86a5-39bf-45d3-a399-56d88697f38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:29.117155 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-e2fd86a5-39bf-45d3-a399-56d88697f38e None None] MFA Rules not processed for user `ef222fe025534a9e942f8c5c4ed1e3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:29.136816 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-e2fd86a5-39bf-45d3-a399-56d88697f38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:29.522395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 853/1711] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 531 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:29.547402 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc646d59-2614-446e-ace0-541c0b0c1f05 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:29.550489 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-dc646d59-2614-446e-ace0-541c0b0c1f05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:30.022295 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc646d59-2614-446e-ace0-541c0b0c1f05 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:30.057887 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4364bb-596f-4c1a-9a5f-7a7c81cb3848 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:30.060123 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4364bb-596f-4c1a-9a5f-7a7c81cb3848 service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:30.068405 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4364bb-596f-4c1a-9a5f-7a7c81cb3848 service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:30.070451 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4364bb-596f-4c1a-9a5f-7a7c81cb3848 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:30.135702 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4364bb-596f-4c1a-9a5f-7a7c81cb3848 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:30.137585 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4364bb-596f-4c1a-9a5f-7a7c81cb3848 service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:30.151336 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98082b81-9a3d-42ea-930d-b2e9f1691f54 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:30.151690 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98082b81-9a3d-42ea-930d-b2e9f1691f54 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:30.151985 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98082b81-9a3d-42ea-930d-b2e9f1691f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:30.221984 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 859/1712] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 675 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:30.273732 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-98082b81-9a3d-42ea-930d-b2e9f1691f54 None None] MFA Rules not processed for user `771d3e22beff44f7bbc3103fd75a35b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:30.280516 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-98082b81-9a3d-42ea-930d-b2e9f1691f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:30.359892 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:30.508350 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4364bb-596f-4c1a-9a5f-7a7c81cb3848 service nova] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:30.510239 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1b4364bb-596f-4c1a-9a5f-7a7c81cb3848 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:30.553125 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 854/1713] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:30.576483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98082b81-9a3d-42ea-930d-b2e9f1691f54 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:30.579126 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-98082b81-9a3d-42ea-930d-b2e9f1691f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:30.616210 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98082b81-9a3d-42ea-930d-b2e9f1691f54 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:30.627622 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea0aba-35bf-4834-bbc6-aad88d88683f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:30.629175 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea0aba-35bf-4834-bbc6-aad88d88683f service nova] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:30.629968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea0aba-35bf-4834-bbc6-aad88d88683f service nova] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:30.630517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea0aba-35bf-4834-bbc6-aad88d88683f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:30.658924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea0aba-35bf-4834-bbc6-aad88d88683f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:30.662763 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea0aba-35bf-4834-bbc6-aad88d88683f service nova] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:30.703941 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 855/1714] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:30.753035 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4364bb-596f-4c1a-9a5f-7a7c81cb3848 service nova] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:30.774989 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c35a51-baa8-4b4e-a8bd-3792215720c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:30.778338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c35a51-baa8-4b4e-a8bd-3792215720c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:30.779872 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c35a51-baa8-4b4e-a8bd-3792215720c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:30.781126 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c35a51-baa8-4b4e-a8bd-3792215720c2 admin admin] PATH_INFO: `/v3/users/60fd535764604783ae128519bf0963e4` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:30.785014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c35a51-baa8-4b4e-a8bd-3792215720c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60fd535764604783ae128519bf0963e4)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:30.796768 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c35a51-baa8-4b4e-a8bd-3792215720c2 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:30.986782 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-54c35a51-baa8-4b4e-a8bd-3792215720c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60fd535764604783ae128519bf0963e4'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:31.048630 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:31.061188 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-54c35a51-baa8-4b4e-a8bd-3792215720c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60fd535764604783ae128519bf0963e4'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:31.083188 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-54c35a51-baa8-4b4e-a8bd-3792215720c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60fd535764604783ae128519bf0963e4'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:31.084108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-54c35a51-baa8-4b4e-a8bd-3792215720c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60fd535764604783ae128519bf0963e4'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:31.092913 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-54c35a51-baa8-4b4e-a8bd-3792215720c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60fd535764604783ae128519bf0963e4'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:31.098410 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea0aba-35bf-4834-bbc6-aad88d88683f service nova] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:31.101956 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-1cea0aba-35bf-4834-bbc6-aad88d88683f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:31.111365 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 860/1715] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:33:30 2022] DELETE /identity/v3/users/60fd535764604783ae128519bf0963e4 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:31.119351 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2336ec-2fbe-4e45-a9ae-cd0f6c7e6e55 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:31.119883 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2336ec-2fbe-4e45-a9ae-cd0f6c7e6e55 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:31.120471 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2336ec-2fbe-4e45-a9ae-cd0f6c7e6e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:31.186177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-ea2336ec-2fbe-4e45-a9ae-cd0f6c7e6e55 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:31.187543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ea2336ec-2fbe-4e45-a9ae-cd0f6c7e6e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:31.217608 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea0aba-35bf-4834-bbc6-aad88d88683f service nova] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:31.370094 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 861/1716] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:33:31.484368 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044e565-52d3-48df-8dd4-d698d7ebb2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:31.485292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044e565-52d3-48df-8dd4-d698d7ebb2d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:31.485583 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044e565-52d3-48df-8dd4-d698d7ebb2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:31.485867 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044e565-52d3-48df-8dd4-d698d7ebb2d9 admin admin] PATH_INFO: `/v3/projects/14767d6407b645efade02ea237a97ede` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:31.489052 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e044e565-52d3-48df-8dd4-d698d7ebb2d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14767d6407b645efade02ea237a97ede)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:31.491448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e044e565-52d3-48df-8dd4-d698d7ebb2d9 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:31.742316 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-e044e565-52d3-48df-8dd4-d698d7ebb2d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14767d6407b645efade02ea237a97ede', 'request_id': 'req-e044e565-52d3-48df-8dd4-d698d7ebb2d9'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:31.778226 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-e044e565-52d3-48df-8dd4-d698d7ebb2d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14767d6407b645efade02ea237a97ede', 'request_id': 'req-e044e565-52d3-48df-8dd4-d698d7ebb2d9'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:31.798836 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-e044e565-52d3-48df-8dd4-d698d7ebb2d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14767d6407b645efade02ea237a97ede', 'request_id': 'req-e044e565-52d3-48df-8dd4-d698d7ebb2d9'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:31.897013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-e044e565-52d3-48df-8dd4-d698d7ebb2d9 admin admin] The token cache is being invalidate because project 14767d6407b645efade02ea237a97ede was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:33:31.897838 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-e044e565-52d3-48df-8dd4-d698d7ebb2d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:31.899469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-e044e565-52d3-48df-8dd4-d698d7ebb2d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:31.905289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 856/1717] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:33:31 2022] DELETE /identity/v3/projects/14767d6407b645efade02ea237a97ede => generated 0 bytes in 810 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:31.911427 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd533fa-f775-4a7a-b6a1-80cee11a5a4d None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:31.911427 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd533fa-f775-4a7a-b6a1-80cee11a5a4d None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:31.912434 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd533fa-f775-4a7a-b6a1-80cee11a5a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:31.991824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-bbd533fa-f775-4a7a-b6a1-80cee11a5a4d None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:31.993128 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-bbd533fa-f775-4a7a-b6a1-80cee11a5a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:32.084079 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ca343-99f3-48d9-98dc-64e425a70163 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:32.084700 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ca343-99f3-48d9-98dc-64e425a70163 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:32.085637 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ca343-99f3-48d9-98dc-64e425a70163 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:32.089750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 857/1718] 10.222.0.16 () {54 vars in 921 bytes} [Mon May 23 12:33:32 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:33:32.100714 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7ca343-99f3-48d9-98dc-64e425a70163 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:32.106108 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-bb7ca343-99f3-48d9-98dc-64e425a70163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:32.251087 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 862/1719] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:32.264118 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa96d6-39f1-45dc-8dd0-1b01f69ad9de None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:32.281377 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa96d6-39f1-45dc-8dd0-1b01f69ad9de None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:32.282547 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa96d6-39f1-45dc-8dd0-1b01f69ad9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:32.390417 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-30aa96d6-39f1-45dc-8dd0-1b01f69ad9de None None] MFA Rules not processed for user `b83535b443a84275a160cd84cb3ad5a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:32.391787 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-30aa96d6-39f1-45dc-8dd0-1b01f69ad9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:32.423213 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:32.467646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7ca343-99f3-48d9-98dc-64e425a70163 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:32.473301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28807bb-41f7-4b5b-8458-0f271fe7b1a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:32.480933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28807bb-41f7-4b5b-8458-0f271fe7b1a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:32.482731 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28807bb-41f7-4b5b-8458-0f271fe7b1a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:32.485501 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28807bb-41f7-4b5b-8458-0f271fe7b1a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:32.494580 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-c28807bb-41f7-4b5b-8458-0f271fe7b1a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:32.549259 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28807bb-41f7-4b5b-8458-0f271fe7b1a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:32.550755 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28807bb-41f7-4b5b-8458-0f271fe7b1a2 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:32.582514 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 858/1720] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:32.584223 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e3c409-57ee-413e-ab63-cfbd3d509555 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:32.597838 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e3c409-57ee-413e-ab63-cfbd3d509555 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:32.598548 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e3c409-57ee-413e-ab63-cfbd3d509555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:32.657919 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-31e3c409-57ee-413e-ab63-cfbd3d509555 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:32.659196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-31e3c409-57ee-413e-ab63-cfbd3d509555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:32.694498 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 859/1721] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:32.699558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e3c409-57ee-413e-ab63-cfbd3d509555 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:32.722213 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e3c409-57ee-413e-ab63-cfbd3d509555 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:32.724932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 863/1722] 10.222.0.16 () {58 vars in 1018 bytes} [Mon May 23 12:33:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:32.744732 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16022b11-2c42-4492-afa1-410af8b0a1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:32.746100 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16022b11-2c42-4492-afa1-410af8b0a1a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:32.746540 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16022b11-2c42-4492-afa1-410af8b0a1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:32.746963 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16022b11-2c42-4492-afa1-410af8b0a1a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:32.747870 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16022b11-2c42-4492-afa1-410af8b0a1a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:33:32.748447 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16022b11-2c42-4492-afa1-410af8b0a1a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:32.750319 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16022b11-2c42-4492-afa1-410af8b0a1a1 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:32.820039 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 860/1723] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:33:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:33:32.827844 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa96d6-39f1-45dc-8dd0-1b01f69ad9de None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:32.865782 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa96d6-39f1-45dc-8dd0-1b01f69ad9de None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:32.889946 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03676ec8-551e-4fc9-b8d9-9fbc7939ccb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:32.890901 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03676ec8-551e-4fc9-b8d9-9fbc7939ccb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:32.893391 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03676ec8-551e-4fc9-b8d9-9fbc7939ccb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:32.893948 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03676ec8-551e-4fc9-b8d9-9fbc7939ccb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:32.895790 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03676ec8-551e-4fc9-b8d9-9fbc7939ccb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:32.905213 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03676ec8-551e-4fc9-b8d9-9fbc7939ccb8 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:32.930650 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:33.015254 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 864/1724] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:33:32 2022] POST /identity/v3/projects => generated 367 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:33.038855 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16022b11-2c42-4492-afa1-410af8b0a1a1 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:33.085896 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16022b11-2c42-4492-afa1-410af8b0a1a1 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:33.371043 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03676ec8-551e-4fc9-b8d9-9fbc7939ccb8 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:33.373119 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-03676ec8-551e-4fc9-b8d9-9fbc7939ccb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:33.413871 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990ad60c-278a-49d0-a5f7-1729338dabdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:33.416259 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ad60c-278a-49d0-a5f7-1729338dabdb admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:33.416667 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ad60c-278a-49d0-a5f7-1729338dabdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:33.416963 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ad60c-278a-49d0-a5f7-1729338dabdb admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:33.417786 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990ad60c-278a-49d0-a5f7-1729338dabdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:33.422849 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990ad60c-278a-49d0-a5f7-1729338dabdb admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:33.514297 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 861/1725] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:33:33 2022] POST /identity/v3/users => generated 284 bytes in 496 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:33.516531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990ad60c-278a-49d0-a5f7-1729338dabdb admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:33.523066 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-990ad60c-278a-49d0-a5f7-1729338dabdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:33.684513 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03676ec8-551e-4fc9-b8d9-9fbc7939ccb8 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:33.704449 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60ae05-1496-4bd0-b26b-4201202f210e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:33.705409 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60ae05-1496-4bd0-b26b-4201202f210e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:33.705697 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60ae05-1496-4bd0-b26b-4201202f210e admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:33.706000 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60ae05-1496-4bd0-b26b-4201202f210e admin admin] PATH_INFO: `/v3/users/00a6d045a92d48799bd91d84bedc383d` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:33.717592 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d60ae05-1496-4bd0-b26b-4201202f210e admin admin] RBAC: Authorizing `identity:delete_user(user_id=00a6d045a92d48799bd91d84bedc383d)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:33.725815 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d60ae05-1496-4bd0-b26b-4201202f210e admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:33.767035 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990ad60c-278a-49d0-a5f7-1729338dabdb admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:33.798759 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61757500-1f3c-4020-9ab2-4248185232e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:33.799483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61757500-1f3c-4020-9ab2-4248185232e0 service nova] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:33.799781 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61757500-1f3c-4020-9ab2-4248185232e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:33.800364 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61757500-1f3c-4020-9ab2-4248185232e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:33.817395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61757500-1f3c-4020-9ab2-4248185232e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:33.819060 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61757500-1f3c-4020-9ab2-4248185232e0 service nova] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:33.867649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 862/1726] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:33.868628 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61757500-1f3c-4020-9ab2-4248185232e0 service nova] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:33.883108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8d60ae05-1496-4bd0-b26b-4201202f210e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00a6d045a92d48799bd91d84bedc383d'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:33.925634 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61757500-1f3c-4020-9ab2-4248185232e0 service nova] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:33.927642 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8d60ae05-1496-4bd0-b26b-4201202f210e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00a6d045a92d48799bd91d84bedc383d'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:33.941342 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ed778-4dd1-49fa-9fbb-ed70af4f5df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:33.942180 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ed778-4dd1-49fa-9fbb-ed70af4f5df0 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:33.942481 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ed778-4dd1-49fa-9fbb-ed70af4f5df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:33.944864 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ed778-4dd1-49fa-9fbb-ed70af4f5df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:33.945749 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ed778-4dd1-49fa-9fbb-ed70af4f5df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:33.950108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8d60ae05-1496-4bd0-b26b-4201202f210e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00a6d045a92d48799bd91d84bedc383d'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:33.951813 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8d60ae05-1496-4bd0-b26b-4201202f210e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00a6d045a92d48799bd91d84bedc383d'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:33.952577 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8d60ae05-1496-4bd0-b26b-4201202f210e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00a6d045a92d48799bd91d84bedc383d'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:33.956516 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ed778-4dd1-49fa-9fbb-ed70af4f5df0 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:33.989836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 865/1727] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:33:33 2022] DELETE /identity/v3/users/00a6d045a92d48799bd91d84bedc383d => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:33:33.995647 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6384106-8bd0-4c5a-95f0-b67571dd3c4f None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:33.996027 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6384106-8bd0-4c5a-95f0-b67571dd3c4f None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:33.996251 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6384106-8bd0-4c5a-95f0-b67571dd3c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:33.998074 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 863/1728] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:34.003346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ed778-4dd1-49fa-9fbb-ed70af4f5df0 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:34.007701 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-de1ed778-4dd1-49fa-9fbb-ed70af4f5df0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:34.058693 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ed778-4dd1-49fa-9fbb-ed70af4f5df0 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:34.068838 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-f6384106-8bd0-4c5a-95f0-b67571dd3c4f None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:34.075208 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-f6384106-8bd0-4c5a-95f0-b67571dd3c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:34.131252 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:34.219285 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2be322c-9d8d-404b-a965-fbc1bd48dc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:34.220029 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be322c-9d8d-404b-a965-fbc1bd48dc84 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:34.221042 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be322c-9d8d-404b-a965-fbc1bd48dc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:34.221333 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be322c-9d8d-404b-a965-fbc1bd48dc84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:34.222550 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2be322c-9d8d-404b-a965-fbc1bd48dc84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:34.226462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2be322c-9d8d-404b-a965-fbc1bd48dc84 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:34.249091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 866/1729] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:34.255382 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 864/1730] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:34.261569 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6384106-8bd0-4c5a-95f0-b67571dd3c4f None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:34.270944 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:34.276922 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6384106-8bd0-4c5a-95f0-b67571dd3c4f None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:34.293413 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1e7fc-aae1-4354-8bc0-2e2164a803d8 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:34.293713 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1e7fc-aae1-4354-8bc0-2e2164a803d8 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:34.294003 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1e7fc-aae1-4354-8bc0-2e2164a803d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:34.299631 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a9dfe-4b66-4a4c-9527-7465d46f9089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:34.303127 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a9dfe-4b66-4a4c-9527-7465d46f9089 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:34.303413 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a9dfe-4b66-4a4c-9527-7465d46f9089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:34.303692 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a9dfe-4b66-4a4c-9527-7465d46f9089 admin admin] PATH_INFO: `/v3/projects/6c1d070c72404e148b5c158e8817cc8e/users/e90cfa6008904c62a51b9d3eab457a11/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:34.344516 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-06f1e7fc-aae1-4354-8bc0-2e2164a803d8 None None] MFA Rules not processed for user `851454212d14438d80bf8e193fea268e`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:34.348881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-06f1e7fc-aae1-4354-8bc0-2e2164a803d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:34.352296 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a9dfe-4b66-4a4c-9527-7465d46f9089 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c1d070c72404e148b5c158e8817cc8e, user_id=e90cfa6008904c62a51b9d3eab457a11, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:34.357622 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a9dfe-4b66-4a4c-9527-7465d46f9089 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:34.420235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 867/1731] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:34 2022] PUT /identity/v3/projects/6c1d070c72404e148b5c158e8817cc8e/users/e90cfa6008904c62a51b9d3eab457a11/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:34.420235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a9dfe-4b66-4a4c-9527-7465d46f9089 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:34.420235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e90a9dfe-4b66-4a4c-9527-7465d46f9089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:34.553468 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 865/1732] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:33:34.556842 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e39c27-881a-4dda-9ee5-6f9f1a67d4dc None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:34.559393 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e39c27-881a-4dda-9ee5-6f9f1a67d4dc None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:34.559814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e39c27-881a-4dda-9ee5-6f9f1a67d4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:34.593175 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a9dfe-4b66-4a4c-9527-7465d46f9089 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:34.599445 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1172d-ba1e-4cac-bdd6-7f9d5ade00f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:34.599853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1172d-ba1e-4cac-bdd6-7f9d5ade00f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:34.600140 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1172d-ba1e-4cac-bdd6-7f9d5ade00f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:34.600446 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b1172d-ba1e-4cac-bdd6-7f9d5ade00f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:34.639565 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-23e39c27-881a-4dda-9ee5-6f9f1a67d4dc None None] MFA Rules not processed for user `e90cfa6008904c62a51b9d3eab457a11`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:34.652975 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-23e39c27-881a-4dda-9ee5-6f9f1a67d4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:34.840440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b1172d-ba1e-4cac-bdd6-7f9d5ade00f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:34.841928 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b1172d-ba1e-4cac-bdd6-7f9d5ade00f9 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:34.849563 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 866/1733] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:34 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:34.851408 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e39c27-881a-4dda-9ee5-6f9f1a67d4dc None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:34.858358 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-23e39c27-881a-4dda-9ee5-6f9f1a67d4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:34.872485 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 868/1734] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:34.873561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1172d-ba1e-4cac-bdd6-7f9d5ade00f9 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:34.893807 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b1172d-ba1e-4cac-bdd6-7f9d5ade00f9 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:34.909997 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0cb46-e9d6-4ec3-9f67-5de9cd8f6bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:34.910689 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0cb46-e9d6-4ec3-9f67-5de9cd8f6bb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:34.910972 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0cb46-e9d6-4ec3-9f67-5de9cd8f6bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:34.911074 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e39c27-881a-4dda-9ee5-6f9f1a67d4dc None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:34.911248 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f0cb46-e9d6-4ec3-9f67-5de9cd8f6bb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:34.911946 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0cb46-e9d6-4ec3-9f67-5de9cd8f6bb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:34.914106 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f0cb46-e9d6-4ec3-9f67-5de9cd8f6bb8 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:34.929064 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad058c1-43c5-4dfa-bd00-ba7993d43cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:34.930165 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad058c1-43c5-4dfa-bd00-ba7993d43cd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:34.930598 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad058c1-43c5-4dfa-bd00-ba7993d43cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:34.931217 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad058c1-43c5-4dfa-bd00-ba7993d43cd5 admin admin] PATH_INFO: `/v3/projects/eb8f810a63bb403ba2f47529aba432a1` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:34.934631 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad058c1-43c5-4dfa-bd00-ba7993d43cd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb8f810a63bb403ba2f47529aba432a1)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:34.938680 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad058c1-43c5-4dfa-bd00-ba7993d43cd5 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:34.974603 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 869/1735] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:33:34 2022] POST /identity/v3/projects => generated 369 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:34.983707 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0cb46-e9d6-4ec3-9f67-5de9cd8f6bb8 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:35.104386 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-fad058c1-43c5-4dfa-bd00-ba7993d43cd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb8f810a63bb403ba2f47529aba432a1', 'request_id': 'req-fad058c1-43c5-4dfa-bd00-ba7993d43cd5'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:35.109208 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:35.121286 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-fad058c1-43c5-4dfa-bd00-ba7993d43cd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb8f810a63bb403ba2f47529aba432a1', 'request_id': 'req-fad058c1-43c5-4dfa-bd00-ba7993d43cd5'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:35.133104 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-fad058c1-43c5-4dfa-bd00-ba7993d43cd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb8f810a63bb403ba2f47529aba432a1', 'request_id': 'req-fad058c1-43c5-4dfa-bd00-ba7993d43cd5'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:35.134655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f0cb46-e9d6-4ec3-9f67-5de9cd8f6bb8 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:35.157241 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-fad058c1-43c5-4dfa-bd00-ba7993d43cd5 admin admin] The token cache is being invalidate because project eb8f810a63bb403ba2f47529aba432a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:33:35.157670 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-fad058c1-43c5-4dfa-bd00-ba7993d43cd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:35.158477 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-fad058c1-43c5-4dfa-bd00-ba7993d43cd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:35.163210 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 867/1736] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:33:34 2022] DELETE /identity/v3/projects/eb8f810a63bb403ba2f47529aba432a1 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:35.165008 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:35.170516 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad058c1-43c5-4dfa-bd00-ba7993d43cd5 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:35.174649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-fad058c1-43c5-4dfa-bd00-ba7993d43cd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:35.277491 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446bdaf-d5b1-445b-85d5-2c346474b5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:35.278812 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446bdaf-d5b1-445b-85d5-2c346474b5ba admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:35.279102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446bdaf-d5b1-445b-85d5-2c346474b5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:35.279585 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446bdaf-d5b1-445b-85d5-2c346474b5ba admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:35.280318 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a446bdaf-d5b1-445b-85d5-2c346474b5ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:35.284035 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a446bdaf-d5b1-445b-85d5-2c346474b5ba admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:35.332383 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 870/1737] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:33:34 2022] POST /identity/v3/users => generated 291 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:35.336321 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446bdaf-d5b1-445b-85d5-2c346474b5ba admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:35.346675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a446bdaf-d5b1-445b-85d5-2c346474b5ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:35.393455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446bdaf-d5b1-445b-85d5-2c346474b5ba admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:35.399570 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad058c1-43c5-4dfa-bd00-ba7993d43cd5 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:35.423469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54623857-746f-4f15-aef3-19979d0810b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:35.424853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54623857-746f-4f15-aef3-19979d0810b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:35.425309 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54623857-746f-4f15-aef3-19979d0810b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:35.425741 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54623857-746f-4f15-aef3-19979d0810b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:35.432609 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54623857-746f-4f15-aef3-19979d0810b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:35.438096 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54623857-746f-4f15-aef3-19979d0810b1 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:35.445178 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a3401-f60c-4bc8-8cb5-7bd7eb9c78a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:35.446095 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a3401-f60c-4bc8-8cb5-7bd7eb9c78a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:35.446388 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a3401-f60c-4bc8-8cb5-7bd7eb9c78a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:35.447432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a3401-f60c-4bc8-8cb5-7bd7eb9c78a6 admin admin] PATH_INFO: `/v3/users/bf7bb760aa03484c859645de28832f72` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:35.459207 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a3401-f60c-4bc8-8cb5-7bd7eb9c78a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf7bb760aa03484c859645de28832f72)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:35.465965 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a3401-f60c-4bc8-8cb5-7bd7eb9c78a6 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:35.491494 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 871/1738] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:35.492595 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54623857-746f-4f15-aef3-19979d0810b1 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:35.516034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54623857-746f-4f15-aef3-19979d0810b1 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:35.536613 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4c3b19-3a7e-4f6e-9ac5-34228b94f96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:35.537536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c3b19-3a7e-4f6e-9ac5-34228b94f96b admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:35.537824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c3b19-3a7e-4f6e-9ac5-34228b94f96b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:35.538097 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c3b19-3a7e-4f6e-9ac5-34228b94f96b admin admin] PATH_INFO: `/v3/projects/5b05cf3cf4084bd6a82d6bcbc339b69e/users/c82bf0e3b9ec41cbb318771f12650990/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:35.606202 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-aa5a3401-f60c-4bc8-8cb5-7bd7eb9c78a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf7bb760aa03484c859645de28832f72'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:35.606513 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-aa5a3401-f60c-4bc8-8cb5-7bd7eb9c78a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf7bb760aa03484c859645de28832f72'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:35.609536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4c3b19-3a7e-4f6e-9ac5-34228b94f96b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b05cf3cf4084bd6a82d6bcbc339b69e, user_id=c82bf0e3b9ec41cbb318771f12650990, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:35.612303 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4c3b19-3a7e-4f6e-9ac5-34228b94f96b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:35.644987 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-aa5a3401-f60c-4bc8-8cb5-7bd7eb9c78a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf7bb760aa03484c859645de28832f72'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:35.660921 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 872/1739] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:35 2022] PUT /identity/v3/projects/5b05cf3cf4084bd6a82d6bcbc339b69e/users/c82bf0e3b9ec41cbb318771f12650990/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:35.676540 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4c3b19-3a7e-4f6e-9ac5-34228b94f96b admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:35.703957 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-aa5a3401-f60c-4bc8-8cb5-7bd7eb9c78a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf7bb760aa03484c859645de28832f72'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:35.725577 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-aa5a3401-f60c-4bc8-8cb5-7bd7eb9c78a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf7bb760aa03484c859645de28832f72'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:35.740295 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 868/1740] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:33:35 2022] DELETE /identity/v3/users/bf7bb760aa03484c859645de28832f72 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:35.918992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4c3b19-3a7e-4f6e-9ac5-34228b94f96b admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:36.060789 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52dacb-b8c4-4891-b413-08febe091601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:36.061884 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52dacb-b8c4-4891-b413-08febe091601 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:36.061884 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52dacb-b8c4-4891-b413-08febe091601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:36.062275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52dacb-b8c4-4891-b413-08febe091601 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:36.063040 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52dacb-b8c4-4891-b413-08febe091601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:36.064992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52dacb-b8c4-4891-b413-08febe091601 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:36.094311 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 873/1741] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:36.097906 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a3401-f60c-4bc8-8cb5-7bd7eb9c78a6 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:36.104155 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-aa5a3401-f60c-4bc8-8cb5-7bd7eb9c78a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:36.131454 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a3401-f60c-4bc8-8cb5-7bd7eb9c78a6 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:36.147738 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52dacb-b8c4-4891-b413-08febe091601 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:36.149552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-dd52dacb-b8c4-4891-b413-08febe091601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:36.151773 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0fe9f-8458-44da-8961-e2eb0b4cd750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:36.152748 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0fe9f-8458-44da-8961-e2eb0b4cd750 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:36.153126 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0fe9f-8458-44da-8961-e2eb0b4cd750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:36.153365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0fe9f-8458-44da-8961-e2eb0b4cd750 admin admin] PATH_INFO: `/v3/projects/5b05cf3cf4084bd6a82d6bcbc339b69e/users/c82bf0e3b9ec41cbb318771f12650990/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:36.159508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0fe9f-8458-44da-8961-e2eb0b4cd750 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b05cf3cf4084bd6a82d6bcbc339b69e, user_id=c82bf0e3b9ec41cbb318771f12650990, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:36.162398 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0fe9f-8458-44da-8961-e2eb0b4cd750 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:36.179766 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52dacb-b8c4-4891-b413-08febe091601 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:36.200762 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf60df5-343f-4528-864f-c094c22acc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:36.203493 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf60df5-343f-4528-864f-c094c22acc41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:36.204147 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf60df5-343f-4528-864f-c094c22acc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:36.204649 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf60df5-343f-4528-864f-c094c22acc41 admin admin] PATH_INFO: `/v3/projects/02c94f61c8fa4f16a11818e663776ac5` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:36.210668 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf60df5-343f-4528-864f-c094c22acc41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02c94f61c8fa4f16a11818e663776ac5)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:36.221256 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 869/1742] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:36 2022] PUT /identity/v3/projects/5b05cf3cf4084bd6a82d6bcbc339b69e/users/c82bf0e3b9ec41cbb318771f12650990/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:36.223065 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bbbc6-7449-46e7-a0c2-03534ba320e4 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:36.223366 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bbbc6-7449-46e7-a0c2-03534ba320e4 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:36.223648 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bbbc6-7449-46e7-a0c2-03534ba320e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:36.224621 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf60df5-343f-4528-864f-c094c22acc41 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:36.236526 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:36.311804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-adf60df5-343f-4528-864f-c094c22acc41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02c94f61c8fa4f16a11818e663776ac5', 'request_id': 'req-adf60df5-343f-4528-864f-c094c22acc41'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:36.324322 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-adf60df5-343f-4528-864f-c094c22acc41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02c94f61c8fa4f16a11818e663776ac5', 'request_id': 'req-adf60df5-343f-4528-864f-c094c22acc41'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:36.335679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-adf60df5-343f-4528-864f-c094c22acc41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02c94f61c8fa4f16a11818e663776ac5', 'request_id': 'req-adf60df5-343f-4528-864f-c094c22acc41'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:36.358236 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-adf60df5-343f-4528-864f-c094c22acc41 admin admin] The token cache is being invalidate because project 02c94f61c8fa4f16a11818e663776ac5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:33:36.358589 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-adf60df5-343f-4528-864f-c094c22acc41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:36.360220 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-adf60df5-343f-4528-864f-c094c22acc41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:36.366851 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 874/1743] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:33:36 2022] DELETE /identity/v3/projects/02c94f61c8fa4f16a11818e663776ac5 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:33:36.370460 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf60df5-343f-4528-864f-c094c22acc41 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:36.372266 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-adf60df5-343f-4528-864f-c094c22acc41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:36.430038 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-4f0bbbc6-7449-46e7-a0c2-03534ba320e4 None None] MFA Rules not processed for user `c82bf0e3b9ec41cbb318771f12650990`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:36.433319 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-4f0bbbc6-7449-46e7-a0c2-03534ba320e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:36.478559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:36.487468 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf60df5-343f-4528-864f-c094c22acc41 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:36.496756 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce4aec-b6af-4cfd-9849-f66957ea9509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:36.497708 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce4aec-b6af-4cfd-9849-f66957ea9509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:36.497998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce4aec-b6af-4cfd-9849-f66957ea9509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:36.498896 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce4aec-b6af-4cfd-9849-f66957ea9509 admin admin] PATH_INFO: `/v3/users/bf53ea54ee20453f9ed1995b1503309d` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:36.500474 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce4aec-b6af-4cfd-9849-f66957ea9509 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf53ea54ee20453f9ed1995b1503309d)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:36.502498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce4aec-b6af-4cfd-9849-f66957ea9509 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:36.557204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 870/1744] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:36 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:36.569438 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bbbc6-7449-46e7-a0c2-03534ba320e4 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:36.576190 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-4f0bbbc6-7449-46e7-a0c2-03534ba320e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:36.580367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4cce4aec-b6af-4cfd-9849-f66957ea9509 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf53ea54ee20453f9ed1995b1503309d'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:36.622226 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4cce4aec-b6af-4cfd-9849-f66957ea9509 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf53ea54ee20453f9ed1995b1503309d'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:36.640016 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4cce4aec-b6af-4cfd-9849-f66957ea9509 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf53ea54ee20453f9ed1995b1503309d'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:36.640866 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4cce4aec-b6af-4cfd-9849-f66957ea9509 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf53ea54ee20453f9ed1995b1503309d'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:36.645270 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4cce4aec-b6af-4cfd-9849-f66957ea9509 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf53ea54ee20453f9ed1995b1503309d'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:36.681048 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 875/1745] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:33:36 2022] DELETE /identity/v3/users/bf53ea54ee20453f9ed1995b1503309d => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:36.711595 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bbbc6-7449-46e7-a0c2-03534ba320e4 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:36.840492 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0a1d9-3afd-4f29-909f-25cdfcadcbd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:36.841230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0a1d9-3afd-4f29-909f-25cdfcadcbd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:36.841523 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0a1d9-3afd-4f29-909f-25cdfcadcbd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:36.841839 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0a1d9-3afd-4f29-909f-25cdfcadcbd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:36.844975 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-17b0a1d9-3afd-4f29-909f-25cdfcadcbd5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:37.017523 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0a1d9-3afd-4f29-909f-25cdfcadcbd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:37.019967 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0a1d9-3afd-4f29-909f-25cdfcadcbd5 service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:37.065386 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 871/1746] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:36 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:37.152768 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce4aec-b6af-4cfd-9849-f66957ea9509 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:37.155269 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-4cce4aec-b6af-4cfd-9849-f66957ea9509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:37.228575 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce4aec-b6af-4cfd-9849-f66957ea9509 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:37.249996 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba078f-f62d-42da-8a59-51ce6bad0346 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:37.252187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba078f-f62d-42da-8a59-51ce6bad0346 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:37.252791 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba078f-f62d-42da-8a59-51ce6bad0346 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:37.253274 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba078f-f62d-42da-8a59-51ce6bad0346 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:37.257169 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e9ba078f-f62d-42da-8a59-51ce6bad0346 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:37.279387 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:37.374520 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba078f-f62d-42da-8a59-51ce6bad0346 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:37.376356 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba078f-f62d-42da-8a59-51ce6bad0346 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:37.393603 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 876/1747] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:37 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:38.184278 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0a1d9-3afd-4f29-909f-25cdfcadcbd5 service cinder] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:38.186358 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-17b0a1d9-3afd-4f29-909f-25cdfcadcbd5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:38.260095 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:38.302957 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0a1d9-3afd-4f29-909f-25cdfcadcbd5 service cinder] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:38.309684 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c8b62-0539-44d7-8e2a-0aa74942e039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:38.310586 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c8b62-0539-44d7-8e2a-0aa74942e039 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:38.310877 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c8b62-0539-44d7-8e2a-0aa74942e039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:38.311157 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c8b62-0539-44d7-8e2a-0aa74942e039 admin admin] PATH_INFO: `/v3/projects/b1619875a23e4d548255a0fcf87cd307` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:38.312457 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c8b62-0539-44d7-8e2a-0aa74942e039 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1619875a23e4d548255a0fcf87cd307)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:38.314506 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c8b62-0539-44d7-8e2a-0aa74942e039 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:38.442095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a502a-0b4b-4c35-98ff-93985290a3cc None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:38.442937 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a502a-0b4b-4c35-98ff-93985290a3cc None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:38.442937 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a502a-0b4b-4c35-98ff-93985290a3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:38.476401 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-ba9c8b62-0539-44d7-8e2a-0aa74942e039 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1619875a23e4d548255a0fcf87cd307', 'request_id': 'req-ba9c8b62-0539-44d7-8e2a-0aa74942e039'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:38.478016 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-992a502a-0b4b-4c35-98ff-93985290a3cc None None] MFA Rules not processed for user `771d3e22beff44f7bbc3103fd75a35b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:38.479429 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-992a502a-0b4b-4c35-98ff-93985290a3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:38.498506 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-ba9c8b62-0539-44d7-8e2a-0aa74942e039 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1619875a23e4d548255a0fcf87cd307', 'request_id': 'req-ba9c8b62-0539-44d7-8e2a-0aa74942e039'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:38.518721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-ba9c8b62-0539-44d7-8e2a-0aa74942e039 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1619875a23e4d548255a0fcf87cd307', 'request_id': 'req-ba9c8b62-0539-44d7-8e2a-0aa74942e039'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:38.549125 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-ba9c8b62-0539-44d7-8e2a-0aa74942e039 admin admin] The token cache is being invalidate because project b1619875a23e4d548255a0fcf87cd307 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:33:38.549696 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-ba9c8b62-0539-44d7-8e2a-0aa74942e039 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:38.550797 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-ba9c8b62-0539-44d7-8e2a-0aa74942e039 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:38.559994 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 872/1748] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:33:38 2022] DELETE /identity/v3/projects/b1619875a23e4d548255a0fcf87cd307 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:38.562515 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7308b7e-3e0a-4801-9077-d809ea01c442 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:38.563028 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7308b7e-3e0a-4801-9077-d809ea01c442 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:38.563514 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7308b7e-3e0a-4801-9077-d809ea01c442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:38.571083 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 877/1749] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:33:38.596921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-f7308b7e-3e0a-4801-9077-d809ea01c442 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:38.603842 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-f7308b7e-3e0a-4801-9077-d809ea01c442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:38.715430 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 873/1750] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:38.724479 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45815db6-fdf2-4c9d-adf0-9d812fd4ce73 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:38.725718 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45815db6-fdf2-4c9d-adf0-9d812fd4ce73 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:38.726112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45815db6-fdf2-4c9d-adf0-9d812fd4ce73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:38.750262 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-45815db6-fdf2-4c9d-adf0-9d812fd4ce73 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:38.751524 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-45815db6-fdf2-4c9d-adf0-9d812fd4ce73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:38.766824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 878/1751] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:38.772048 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7308b7e-3e0a-4801-9077-d809ea01c442 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:38.793456 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7308b7e-3e0a-4801-9077-d809ea01c442 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:38.798548 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ecff24-7050-4cb6-bbb5-8dd300088486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:38.799976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecff24-7050-4cb6-bbb5-8dd300088486 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:38.799976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecff24-7050-4cb6-bbb5-8dd300088486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:38.800131 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecff24-7050-4cb6-bbb5-8dd300088486 admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:38.801110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ecff24-7050-4cb6-bbb5-8dd300088486 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:33:38.801848 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ecff24-7050-4cb6-bbb5-8dd300088486 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:38.803922 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ecff24-7050-4cb6-bbb5-8dd300088486 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:38.820333 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 874/1752] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:33:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:33:38.828112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45815db6-fdf2-4c9d-adf0-9d812fd4ce73 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:38.845629 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45815db6-fdf2-4c9d-adf0-9d812fd4ce73 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:38.857089 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b707b-ab86-428d-a32b-af877ca1c1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:38.858301 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b707b-ab86-428d-a32b-af877ca1c1e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:38.858796 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b707b-ab86-428d-a32b-af877ca1c1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:38.859465 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b707b-ab86-428d-a32b-af877ca1c1e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:38.861442 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883b707b-ab86-428d-a32b-af877ca1c1e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:38.864527 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883b707b-ab86-428d-a32b-af877ca1c1e4 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:38.920518 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab18ed-0998-494a-9736-af37cddaa13c None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:38.921358 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab18ed-0998-494a-9736-af37cddaa13c None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:38.921965 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab18ed-0998-494a-9736-af37cddaa13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:38.932907 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 879/1753] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:33:38 2022] POST /identity/v3/projects => generated 375 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:38.938006 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b707b-ab86-428d-a32b-af877ca1c1e4 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:38.968870 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-12ab18ed-0998-494a-9736-af37cddaa13c None None] MFA Rules not processed for user `687208d733174bffa2783e6857401f39`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:38.970571 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-12ab18ed-0998-494a-9736-af37cddaa13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:39.066127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:39.100012 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b707b-ab86-428d-a32b-af877ca1c1e4 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:39.110873 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52237cce-9e3f-4c59-8bb1-d6d8031d34eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:39.110873 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52237cce-9e3f-4c59-8bb1-d6d8031d34eb admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:39.110873 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52237cce-9e3f-4c59-8bb1-d6d8031d34eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:39.110873 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52237cce-9e3f-4c59-8bb1-d6d8031d34eb admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:39.111218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52237cce-9e3f-4c59-8bb1-d6d8031d34eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:39.112833 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52237cce-9e3f-4c59-8bb1-d6d8031d34eb admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:39.140751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 875/1754] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:33:39.140751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab18ed-0998-494a-9736-af37cddaa13c None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:39.140751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-12ab18ed-0998-494a-9736-af37cddaa13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:39.156918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 880/1755] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:33:38 2022] POST /identity/v3/users => generated 288 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:39.159566 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee619e3-8973-4b63-a336-51433f39a85d None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:39.161269 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee619e3-8973-4b63-a336-51433f39a85d None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:39.161269 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee619e3-8973-4b63-a336-51433f39a85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:39.189384 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab18ed-0998-494a-9736-af37cddaa13c None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:39.204815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc933b3e-4ac8-4e23-9ad8-0e4b297e6694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:39.206000 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc933b3e-4ac8-4e23-9ad8-0e4b297e6694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:39.206552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc933b3e-4ac8-4e23-9ad8-0e4b297e6694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:39.207639 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc933b3e-4ac8-4e23-9ad8-0e4b297e6694 admin admin] PATH_INFO: `/v3/projects/9bce65ee673645868cf2cc29e0d81351` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:39.210212 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc933b3e-4ac8-4e23-9ad8-0e4b297e6694 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bce65ee673645868cf2cc29e0d81351)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:39.216601 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc933b3e-4ac8-4e23-9ad8-0e4b297e6694 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:39.277456 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-8ee619e3-8973-4b63-a336-51433f39a85d None None] MFA Rules not processed for user `687208d733174bffa2783e6857401f39`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:39.279507 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-8ee619e3-8973-4b63-a336-51433f39a85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:39.323788 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 881/1756] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:39.327394 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee619e3-8973-4b63-a336-51433f39a85d None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:39.374397 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee619e3-8973-4b63-a336-51433f39a85d None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:39.384677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bc933b3e-4ac8-4e23-9ad8-0e4b297e6694 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bce65ee673645868cf2cc29e0d81351', 'request_id': 'req-bc933b3e-4ac8-4e23-9ad8-0e4b297e6694'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:39.398911 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:39.411435 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bc933b3e-4ac8-4e23-9ad8-0e4b297e6694 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bce65ee673645868cf2cc29e0d81351', 'request_id': 'req-bc933b3e-4ac8-4e23-9ad8-0e4b297e6694'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:39.422383 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bc933b3e-4ac8-4e23-9ad8-0e4b297e6694 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bce65ee673645868cf2cc29e0d81351', 'request_id': 'req-bc933b3e-4ac8-4e23-9ad8-0e4b297e6694'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:39.439745 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bc933b3e-4ac8-4e23-9ad8-0e4b297e6694 admin admin] The token cache is being invalidate because project 9bce65ee673645868cf2cc29e0d81351 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:33:39.440467 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bc933b3e-4ac8-4e23-9ad8-0e4b297e6694 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:39.441430 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bc933b3e-4ac8-4e23-9ad8-0e4b297e6694 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:39.446985 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 876/1757] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:33:39 2022] DELETE /identity/v3/projects/9bce65ee673645868cf2cc29e0d81351 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:39.460003 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0edc4-7b86-48ec-9eb0-6255b7203b00 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:39.460003 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0edc4-7b86-48ec-9eb0-6255b7203b00 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:39.460003 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0edc4-7b86-48ec-9eb0-6255b7203b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:39.478364 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-6af0edc4-7b86-48ec-9eb0-6255b7203b00 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:39.479649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-6af0edc4-7b86-48ec-9eb0-6255b7203b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:39.517027 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12856c2d-808b-4f0d-a108-77556bbfa7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:39.517734 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12856c2d-808b-4f0d-a108-77556bbfa7bd admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:39.518278 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12856c2d-808b-4f0d-a108-77556bbfa7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:39.518565 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12856c2d-808b-4f0d-a108-77556bbfa7bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:39.519309 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12856c2d-808b-4f0d-a108-77556bbfa7bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:39.524257 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12856c2d-808b-4f0d-a108-77556bbfa7bd admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:39.551645 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 882/1758] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:39.556277 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12856c2d-808b-4f0d-a108-77556bbfa7bd admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:39.562994 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-12856c2d-808b-4f0d-a108-77556bbfa7bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:39.596865 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12856c2d-808b-4f0d-a108-77556bbfa7bd admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:39.619450 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc769fd-394f-4c04-8374-4ae60d794a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:39.620359 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc769fd-394f-4c04-8374-4ae60d794a09 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:39.620646 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc769fd-394f-4c04-8374-4ae60d794a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:39.620929 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc769fd-394f-4c04-8374-4ae60d794a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:39.621896 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc769fd-394f-4c04-8374-4ae60d794a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:39.623508 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc769fd-394f-4c04-8374-4ae60d794a09 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:39.631510 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 877/1759] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:39.635538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0edc4-7b86-48ec-9eb0-6255b7203b00 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:39.638294 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-6af0edc4-7b86-48ec-9eb0-6255b7203b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:39.668243 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 883/1760] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:39.672405 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60d282-6657-41df-b2e4-c16f0347932a None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:39.672696 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60d282-6657-41df-b2e4-c16f0347932a None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:39.672981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60d282-6657-41df-b2e4-c16f0347932a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:39.734698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-fb60d282-6657-41df-b2e4-c16f0347932a None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:39.736550 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-fb60d282-6657-41df-b2e4-c16f0347932a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:39.745937 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 884/1761] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:39.747349 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60d282-6657-41df-b2e4-c16f0347932a None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:39.757552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60d282-6657-41df-b2e4-c16f0347932a None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:39.766897 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da952d-c1f4-47ca-89fb-d12be9f498d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:39.767860 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da952d-c1f4-47ca-89fb-d12be9f498d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:39.768106 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da952d-c1f4-47ca-89fb-d12be9f498d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:39.769739 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da952d-c1f4-47ca-89fb-d12be9f498d9 admin admin] PATH_INFO: `/v3/projects/00dc00f3b98e4ca6878b1a89c7eda087/users/4314899eb1804171923230964700dfb6/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:39.799683 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89da952d-c1f4-47ca-89fb-d12be9f498d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00dc00f3b98e4ca6878b1a89c7eda087, user_id=4314899eb1804171923230964700dfb6, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:39.804365 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89da952d-c1f4-47ca-89fb-d12be9f498d9 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:39.821393 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0edc4-7b86-48ec-9eb0-6255b7203b00 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:39.827570 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:39.845118 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 885/1762] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:39 2022] PUT /identity/v3/projects/00dc00f3b98e4ca6878b1a89c7eda087/users/4314899eb1804171923230964700dfb6/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:39.858538 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da952d-c1f4-47ca-89fb-d12be9f498d9 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:40.020230 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da952d-c1f4-47ca-89fb-d12be9f498d9 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:40.028228 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f39a7d-ec7c-4eb3-b643-7d9b5e875a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:40.028417 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0788d2-1478-4d4c-a586-f8262dfd0259 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:40.029020 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0788d2-1478-4d4c-a586-f8262dfd0259 service nova] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:40.029305 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0788d2-1478-4d4c-a586-f8262dfd0259 service nova] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:40.029586 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0788d2-1478-4d4c-a586-f8262dfd0259 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:40.030364 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f39a7d-ec7c-4eb3-b643-7d9b5e875a88 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:40.031374 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f39a7d-ec7c-4eb3-b643-7d9b5e875a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:40.032160 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-7e0788d2-1478-4d4c-a586-f8262dfd0259 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:40.034158 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f39a7d-ec7c-4eb3-b643-7d9b5e875a88 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:40.035540 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f39a7d-ec7c-4eb3-b643-7d9b5e875a88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:33:40.037165 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f39a7d-ec7c-4eb3-b643-7d9b5e875a88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:40.041329 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f39a7d-ec7c-4eb3-b643-7d9b5e875a88 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:40.077085 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 886/1763] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:33:40.082985 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd44ddf-6a44-4596-ba9d-02dc14627af1 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:40.083545 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd44ddf-6a44-4596-ba9d-02dc14627af1 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:40.084333 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd44ddf-6a44-4596-ba9d-02dc14627af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:40.121113 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0788d2-1478-4d4c-a586-f8262dfd0259 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:40.122956 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0788d2-1478-4d4c-a586-f8262dfd0259 service nova] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:40.138847 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 878/1764] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:40.139870 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0788d2-1478-4d4c-a586-f8262dfd0259 service nova] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:40.158269 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0788d2-1478-4d4c-a586-f8262dfd0259 service nova] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:40.166416 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dea584-9b42-4ab3-9ccf-05e4e08aa559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:40.167198 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dea584-9b42-4ab3-9ccf-05e4e08aa559 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:40.167667 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dea584-9b42-4ab3-9ccf-05e4e08aa559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:40.168089 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dea584-9b42-4ab3-9ccf-05e4e08aa559 admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:40.168873 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dea584-9b42-4ab3-9ccf-05e4e08aa559 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:40.171260 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dea584-9b42-4ab3-9ccf-05e4e08aa559 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:40.211039 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-cdd44ddf-6a44-4596-ba9d-02dc14627af1 None None] MFA Rules not processed for user `4314899eb1804171923230964700dfb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:40.214425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-cdd44ddf-6a44-4596-ba9d-02dc14627af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:40.226965 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 879/1765] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:33:40 2022] POST /identity/v3/projects => generated 365 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:40.229873 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435c24a-bca6-4a4b-b9d8-2648056beaad None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:40.230170 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435c24a-bca6-4a4b-b9d8-2648056beaad None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:40.230542 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435c24a-bca6-4a4b-b9d8-2648056beaad None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:40.232238 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 880/1766] 10.222.0.16 () {54 vars in 912 bytes} [Mon May 23 12:33:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:33:40.238074 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435c24a-bca6-4a4b-b9d8-2648056beaad None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:40.345188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435c24a-bca6-4a4b-b9d8-2648056beaad None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:40.355962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b5658-ed07-45c0-b58c-312b076bb594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:40.357181 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b5658-ed07-45c0-b58c-312b076bb594 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:40.357181 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b5658-ed07-45c0-b58c-312b076bb594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:40.357498 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b5658-ed07-45c0-b58c-312b076bb594 admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:40.358433 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b5658-ed07-45c0-b58c-312b076bb594 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:40.363744 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b5658-ed07-45c0-b58c-312b076bb594 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:40.382408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:40.430940 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 887/1767] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:40.432530 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3cc77-3aed-4541-ad6c-26becc8355de None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:40.432823 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3cc77-3aed-4541-ad6c-26becc8355de None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:40.433102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3cc77-3aed-4541-ad6c-26becc8355de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:40.454972 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 881/1768] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:33:40 2022] POST /identity/v3/users => generated 283 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:40.455948 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b5658-ed07-45c0-b58c-312b076bb594 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:40.491302 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b5658-ed07-45c0-b58c-312b076bb594 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:40.499083 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a0a408-1508-4791-831f-d12a3fea6f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:40.499550 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a0a408-1508-4791-831f-d12a3fea6f38 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:40.499836 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a0a408-1508-4791-831f-d12a3fea6f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:40.500122 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a0a408-1508-4791-831f-d12a3fea6f38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:40.501251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a0a408-1508-4791-831f-d12a3fea6f38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:40.502994 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a0a408-1508-4791-831f-d12a3fea6f38 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:40.503826 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-c9b3cc77-3aed-4541-ad6c-26becc8355de None None] MFA Rules not processed for user `b6677be3ddf647e2b971bb3ba2c34a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:40.505871 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c9b3cc77-3aed-4541-ad6c-26becc8355de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:40.514607 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:40.544817 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 882/1769] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:33:40 2022] POST /identity/v3/projects => generated 377 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:40.545810 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a0a408-1508-4791-831f-d12a3fea6f38 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:40.626349 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 888/1770] 10.222.0.16 () {58 vars in 1009 bytes} [Mon May 23 12:33:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:40.627273 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3cc77-3aed-4541-ad6c-26becc8355de None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:40.641529 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3cc77-3aed-4541-ad6c-26becc8355de None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:40.650476 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1432bc0-6ff0-462d-9fff-b82bcd50534b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:40.651070 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1432bc0-6ff0-462d-9fff-b82bcd50534b admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:40.651482 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1432bc0-6ff0-462d-9fff-b82bcd50534b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:40.651841 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1432bc0-6ff0-462d-9fff-b82bcd50534b admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:40.652768 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1432bc0-6ff0-462d-9fff-b82bcd50534b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:40.656633 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1432bc0-6ff0-462d-9fff-b82bcd50534b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:40.660298 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a0a408-1508-4791-831f-d12a3fea6f38 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:40.674279 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197db175-6822-4c7b-aaa3-998a6c08a6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:40.675089 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197db175-6822-4c7b-aaa3-998a6c08a6d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:40.675558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197db175-6822-4c7b-aaa3-998a6c08a6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:40.675909 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197db175-6822-4c7b-aaa3-998a6c08a6d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:40.676986 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197db175-6822-4c7b-aaa3-998a6c08a6d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:40.679531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197db175-6822-4c7b-aaa3-998a6c08a6d2 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:40.699639 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 889/1771] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:33:40 2022] POST /identity/v3/users => generated 289 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:40.703616 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1432bc0-6ff0-462d-9fff-b82bcd50534b admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:40.714996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 883/1772] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:40.719334 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197db175-6822-4c7b-aaa3-998a6c08a6d2 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:40.725469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1432bc0-6ff0-462d-9fff-b82bcd50534b admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:40.736890 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478383e-cfbb-4f49-b3be-6ba7e960df41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:40.737894 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478383e-cfbb-4f49-b3be-6ba7e960df41 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:40.741168 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478383e-cfbb-4f49-b3be-6ba7e960df41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:40.742116 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8478383e-cfbb-4f49-b3be-6ba7e960df41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:40.742693 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8478383e-cfbb-4f49-b3be-6ba7e960df41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:40.745477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8478383e-cfbb-4f49-b3be-6ba7e960df41 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:40.748451 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197db175-6822-4c7b-aaa3-998a6c08a6d2 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:40.763415 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505addf-7c89-4157-8fd7-798e85118898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:40.766096 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505addf-7c89-4157-8fd7-798e85118898 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:40.766476 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505addf-7c89-4157-8fd7-798e85118898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:40.766819 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505addf-7c89-4157-8fd7-798e85118898 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:40.772269 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 890/1773] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:40.772453 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505addf-7c89-4157-8fd7-798e85118898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:40.774094 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478383e-cfbb-4f49-b3be-6ba7e960df41 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:40.775538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505addf-7c89-4157-8fd7-798e85118898 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:40.801638 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 884/1774] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:40.806734 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8478383e-cfbb-4f49-b3be-6ba7e960df41 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:40.807023 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505addf-7c89-4157-8fd7-798e85118898 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:40.809360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-d505addf-7c89-4157-8fd7-798e85118898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:40.812667 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78777578-9d58-406c-9c16-b74a039bb8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:40.813562 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78777578-9d58-406c-9c16-b74a039bb8ee admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:40.813904 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78777578-9d58-406c-9c16-b74a039bb8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:40.814346 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78777578-9d58-406c-9c16-b74a039bb8ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:40.815838 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78777578-9d58-406c-9c16-b74a039bb8ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:40.817534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78777578-9d58-406c-9c16-b74a039bb8ee admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:40.838836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 891/1775] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:40.838836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78777578-9d58-406c-9c16-b74a039bb8ee admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:40.875407 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78777578-9d58-406c-9c16-b74a039bb8ee admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:40.895764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae37d7d-c490-4620-834e-d557d5e64e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:40.896933 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae37d7d-c490-4620-834e-d557d5e64e95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:40.897219 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae37d7d-c490-4620-834e-d557d5e64e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:40.897519 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae37d7d-c490-4620-834e-d557d5e64e95 admin admin] PATH_INFO: `/v3/projects/be5dc248c0764095a62067552b30f6d4/users/48f07080d7e8430d839c72895a46db55/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:40.940627 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae37d7d-c490-4620-834e-d557d5e64e95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be5dc248c0764095a62067552b30f6d4, user_id=48f07080d7e8430d839c72895a46db55, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:40.943879 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505addf-7c89-4157-8fd7-798e85118898 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:40.947337 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae37d7d-c490-4620-834e-d557d5e64e95 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:40.955823 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af6ba09-d005-44d7-80cb-1c5504869304 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:40.956736 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af6ba09-d005-44d7-80cb-1c5504869304 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:40.957026 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af6ba09-d005-44d7-80cb-1c5504869304 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:40.957309 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af6ba09-d005-44d7-80cb-1c5504869304 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:40.968446 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af6ba09-d005-44d7-80cb-1c5504869304 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:40.968654 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 892/1776] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:40 2022] PUT /identity/v3/projects/be5dc248c0764095a62067552b30f6d4/users/48f07080d7e8430d839c72895a46db55/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:40.969982 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae37d7d-c490-4620-834e-d557d5e64e95 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:40.980801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af6ba09-d005-44d7-80cb-1c5504869304 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:40.994116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 885/1777] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:40.996029 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1799e-289f-4495-82f5-d07dc495ec8a None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:40.996442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1799e-289f-4495-82f5-d07dc495ec8a None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:40.996786 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1799e-289f-4495-82f5-d07dc495ec8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:41.043030 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-82f1799e-289f-4495-82f5-d07dc495ec8a None None] MFA Rules not processed for user `48f07080d7e8430d839c72895a46db55`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:41.044475 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-82f1799e-289f-4495-82f5-d07dc495ec8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:41.189523 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae37d7d-c490-4620-834e-d557d5e64e95 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:41.196463 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c7f68-a430-4505-a8df-d0c9d4fe9d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:41.197592 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c7f68-a430-4505-a8df-d0c9d4fe9d37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:41.197881 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c7f68-a430-4505-a8df-d0c9d4fe9d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:41.198162 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024c7f68-a430-4505-a8df-d0c9d4fe9d37 admin admin] PATH_INFO: `/v3/projects/76eabfbdf6b546a383e8619acd9e0881/users/6474b34465a94ce7b5898123d055ccab/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:41.206508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 886/1778] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:40 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:41.206508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e743079-3654-48b9-a4e9-3f5bca4e226a None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:41.206508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e743079-3654-48b9-a4e9-3f5bca4e226a None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:41.206822 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e743079-3654-48b9-a4e9-3f5bca4e226a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:41.233587 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:41.236724 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c7f68-a430-4505-a8df-d0c9d4fe9d37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76eabfbdf6b546a383e8619acd9e0881, user_id=6474b34465a94ce7b5898123d055ccab, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:41.239557 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024c7f68-a430-4505-a8df-d0c9d4fe9d37 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:41.251421 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-2e743079-3654-48b9-a4e9-3f5bca4e226a None None] MFA Rules not processed for user `ef222fe025534a9e942f8c5c4ed1e3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:41.252772 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-2e743079-3654-48b9-a4e9-3f5bca4e226a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:41.281573 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 893/1779] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:40 2022] PUT /identity/v3/projects/76eabfbdf6b546a383e8619acd9e0881/users/6474b34465a94ce7b5898123d055ccab/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:41.282613 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c7f68-a430-4505-a8df-d0c9d4fe9d37 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:41.360120 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:41.445923 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 887/1780] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:41.445923 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bedcc2-cd72-4ab2-8ac7-2e047e4c5a12 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:41.446220 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bedcc2-cd72-4ab2-8ac7-2e047e4c5a12 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:41.447582 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bedcc2-cd72-4ab2-8ac7-2e047e4c5a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:41.475440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024c7f68-a430-4505-a8df-d0c9d4fe9d37 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:41.490719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bb06f-585e-49de-9e9f-d4f212beea82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:41.491430 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bb06f-585e-49de-9e9f-d4f212beea82 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:41.491719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bb06f-585e-49de-9e9f-d4f212beea82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:41.492844 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bb06f-585e-49de-9e9f-d4f212beea82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:41.494241 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14bb06f-585e-49de-9e9f-d4f212beea82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:41.496760 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14bb06f-585e-49de-9e9f-d4f212beea82 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:41.518772 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-71bedcc2-cd72-4ab2-8ac7-2e047e4c5a12 None None] MFA Rules not processed for user `6474b34465a94ce7b5898123d055ccab`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:41.526657 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-71bedcc2-cd72-4ab2-8ac7-2e047e4c5a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:41.552038 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 894/1781] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:33:41 2022] POST /identity/v3/projects => generated 363 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:41.558687 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bb06f-585e-49de-9e9f-d4f212beea82 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:41.717343 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 888/1782] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:41.718352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bedcc2-cd72-4ab2-8ac7-2e047e4c5a12 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:41.737067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bb06f-585e-49de-9e9f-d4f212beea82 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:41.742631 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bedcc2-cd72-4ab2-8ac7-2e047e4c5a12 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:41.749999 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2f82f-f5d7-49c7-8faa-3580b5f4a38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:41.750699 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2f82f-f5d7-49c7-8faa-3580b5f4a38f admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:41.751028 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2f82f-f5d7-49c7-8faa-3580b5f4a38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:41.751311 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2f82f-f5d7-49c7-8faa-3580b5f4a38f admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:41.752797 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e2f82f-f5d7-49c7-8faa-3580b5f4a38f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:41.755446 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e2f82f-f5d7-49c7-8faa-3580b5f4a38f admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:41.758431 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d32dcd-42ab-4e34-86e2-152a3ebcd865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:41.759811 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d32dcd-42ab-4e34-86e2-152a3ebcd865 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:41.760132 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d32dcd-42ab-4e34-86e2-152a3ebcd865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:41.760504 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d32dcd-42ab-4e34-86e2-152a3ebcd865 admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:41.761262 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d32dcd-42ab-4e34-86e2-152a3ebcd865 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:41.766015 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d32dcd-42ab-4e34-86e2-152a3ebcd865 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:41.803494 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 895/1783] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:33:41 2022] POST /identity/v3/users => generated 288 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:41.807806 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2f82f-f5d7-49c7-8faa-3580b5f4a38f admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:41.845209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 889/1784] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:33:41 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:41.856611 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d32dcd-42ab-4e34-86e2-152a3ebcd865 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:41.859491 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2f82f-f5d7-49c7-8faa-3580b5f4a38f admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:42.056703 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53fe73c-0e71-46de-90cb-6377ac119c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:42.057703 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53fe73c-0e71-46de-90cb-6377ac119c3b admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:42.060507 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53fe73c-0e71-46de-90cb-6377ac119c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:42.061956 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53fe73c-0e71-46de-90cb-6377ac119c3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:42.062986 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53fe73c-0e71-46de-90cb-6377ac119c3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:42.064396 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d32dcd-42ab-4e34-86e2-152a3ebcd865 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:42.067759 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53fe73c-0e71-46de-90cb-6377ac119c3b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:42.078491 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290eab5-290a-4276-9e95-9a4f4e658b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:42.079549 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290eab5-290a-4276-9e95-9a4f4e658b81 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:42.081348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290eab5-290a-4276-9e95-9a4f4e658b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:42.081811 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290eab5-290a-4276-9e95-9a4f4e658b81 admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:42.085245 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6290eab5-290a-4276-9e95-9a4f4e658b81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:42.091111 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6290eab5-290a-4276-9e95-9a4f4e658b81 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:42.098714 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 896/1785] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:42.107011 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53fe73c-0e71-46de-90cb-6377ac119c3b admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:42.134766 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53fe73c-0e71-46de-90cb-6377ac119c3b admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:42.142102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 890/1786] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:33:41 2022] POST /identity/v3/users => generated 294 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:42.147472 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1bd2ec-56fa-40cf-8608-4c97bf2a0a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:42.148385 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1bd2ec-56fa-40cf-8608-4c97bf2a0a3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:42.149655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1bd2ec-56fa-40cf-8608-4c97bf2a0a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:42.150295 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1bd2ec-56fa-40cf-8608-4c97bf2a0a3e admin admin] PATH_INFO: `/v3/projects/0a9fe3c7c3c94f008ac7788c42894376/users/387e0ebce1cf42fca61102ce56070f9b/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:42.153968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91573be-0924-4e7a-850f-0dade0fe60ed None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:42.154541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91573be-0924-4e7a-850f-0dade0fe60ed None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:42.155445 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91573be-0924-4e7a-850f-0dade0fe60ed None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:42.158103 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 891/1787] 10.222.0.16 () {54 vars in 921 bytes} [Mon May 23 12:33:42 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:33:42.159099 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91573be-0924-4e7a-850f-0dade0fe60ed None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:42.182013 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1bd2ec-56fa-40cf-8608-4c97bf2a0a3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a9fe3c7c3c94f008ac7788c42894376, user_id=387e0ebce1cf42fca61102ce56070f9b, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:42.189169 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91573be-0924-4e7a-850f-0dade0fe60ed None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:42.192424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1bd2ec-56fa-40cf-8608-4c97bf2a0a3e admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:42.202524 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b39a2-7891-4278-9e29-5981995ddfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:42.203226 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b39a2-7891-4278-9e29-5981995ddfeb admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:42.203774 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b39a2-7891-4278-9e29-5981995ddfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:42.204065 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b39a2-7891-4278-9e29-5981995ddfeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:42.205484 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b39a2-7891-4278-9e29-5981995ddfeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:42.207393 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b39a2-7891-4278-9e29-5981995ddfeb admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:42.211628 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:42.231833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 892/1788] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:42.231833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64dd6c-c2e4-4996-8e60-b25b28e4a287 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:42.231833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64dd6c-c2e4-4996-8e60-b25b28e4a287 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:42.231833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64dd6c-c2e4-4996-8e60-b25b28e4a287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:42.236940 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 897/1789] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:42 2022] PUT /identity/v3/projects/0a9fe3c7c3c94f008ac7788c42894376/users/387e0ebce1cf42fca61102ce56070f9b/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:42.239712 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1bd2ec-56fa-40cf-8608-4c97bf2a0a3e admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:42.262719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:42.296230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-4c64dd6c-c2e4-4996-8e60-b25b28e4a287 None None] MFA Rules not processed for user `b83535b443a84275a160cd84cb3ad5a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:42.297512 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-4c64dd6c-c2e4-4996-8e60-b25b28e4a287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:42.395511 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1bd2ec-56fa-40cf-8608-4c97bf2a0a3e admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:42.403071 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873788a4-d337-4d58-ad06-0960040519e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:42.404639 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873788a4-d337-4d58-ad06-0960040519e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:42.404639 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873788a4-d337-4d58-ad06-0960040519e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:42.404639 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873788a4-d337-4d58-ad06-0960040519e2 admin admin] PATH_INFO: `/v3/projects/3fe2b414d769416b834baf5c97ef6fc5/users/15ec6365d3634355b1261ed00bcbc279/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:42.426462 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873788a4-d337-4d58-ad06-0960040519e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fe2b414d769416b834baf5c97ef6fc5, user_id=15ec6365d3634355b1261ed00bcbc279, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:42.433282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 893/1790] 10.222.0.16 () {58 vars in 1018 bytes} [Mon May 23 12:33:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:42.434174 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873788a4-d337-4d58-ad06-0960040519e2 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:42.434284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64dd6c-c2e4-4996-8e60-b25b28e4a287 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:42.450335 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64dd6c-c2e4-4996-8e60-b25b28e4a287 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:42.457610 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 898/1791] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:42 2022] PUT /identity/v3/projects/3fe2b414d769416b834baf5c97ef6fc5/users/15ec6365d3634355b1261ed00bcbc279/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:42.462219 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873788a4-d337-4d58-ad06-0960040519e2 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:42.534959 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11852916-fda0-4ed8-bc66-d13070e32862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:42.535657 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11852916-fda0-4ed8-bc66-d13070e32862 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:42.536377 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11852916-fda0-4ed8-bc66-d13070e32862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:42.536677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11852916-fda0-4ed8-bc66-d13070e32862 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:42.537433 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11852916-fda0-4ed8-bc66-d13070e32862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:42.539408 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11852916-fda0-4ed8-bc66-d13070e32862 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:42.554164 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 894/1792] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:42.555052 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11852916-fda0-4ed8-bc66-d13070e32862 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:42.557189 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-11852916-fda0-4ed8-bc66-d13070e32862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:42.618653 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11852916-fda0-4ed8-bc66-d13070e32862 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:42.627604 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873788a4-d337-4d58-ad06-0960040519e2 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:42.628147 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e830a1-c228-4659-ad31-7eaee82032c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:42.630141 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e830a1-c228-4659-ad31-7eaee82032c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:42.630447 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e830a1-c228-4659-ad31-7eaee82032c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:42.631144 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e830a1-c228-4659-ad31-7eaee82032c9 admin admin] PATH_INFO: `/v3/users/e90cfa6008904c62a51b9d3eab457a11` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:42.632779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e830a1-c228-4659-ad31-7eaee82032c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e90cfa6008904c62a51b9d3eab457a11)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:42.636510 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e830a1-c228-4659-ad31-7eaee82032c9 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:42.640469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757db510-b96f-48ec-9c26-b84af1de4710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:42.642222 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757db510-b96f-48ec-9c26-b84af1de4710 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:42.642512 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757db510-b96f-48ec-9c26-b84af1de4710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:42.642792 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757db510-b96f-48ec-9c26-b84af1de4710 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:42.648569 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757db510-b96f-48ec-9c26-b84af1de4710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:42.653087 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757db510-b96f-48ec-9c26-b84af1de4710 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:42.682208 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 899/1793] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:33:42.683578 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757db510-b96f-48ec-9c26-b84af1de4710 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:42.738889 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757db510-b96f-48ec-9c26-b84af1de4710 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:42.766626 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5b8d5-01ed-45ab-a153-d311dd1b946a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:42.771634 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5b8d5-01ed-45ab-a153-d311dd1b946a admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:42.774483 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5b8d5-01ed-45ab-a153-d311dd1b946a admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:42.785002 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5b8d5-01ed-45ab-a153-d311dd1b946a admin admin] PATH_INFO: `/v3/projects/0a9fe3c7c3c94f008ac7788c42894376/users/387e0ebce1cf42fca61102ce56070f9b/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:42.797119 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5b8d5-01ed-45ab-a153-d311dd1b946a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a9fe3c7c3c94f008ac7788c42894376, user_id=387e0ebce1cf42fca61102ce56070f9b, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:42.800617 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5b8d5-01ed-45ab-a153-d311dd1b946a admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:42.827906 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-20e830a1-c228-4659-ad31-7eaee82032c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e90cfa6008904c62a51b9d3eab457a11'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:42.830226 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-20e830a1-c228-4659-ad31-7eaee82032c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e90cfa6008904c62a51b9d3eab457a11'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:42.857032 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 900/1794] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:42 2022] PUT /identity/v3/projects/0a9fe3c7c3c94f008ac7788c42894376/users/387e0ebce1cf42fca61102ce56070f9b/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:42.858043 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5b8d5-01ed-45ab-a153-d311dd1b946a admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:42.859620 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-20e830a1-c228-4659-ad31-7eaee82032c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e90cfa6008904c62a51b9d3eab457a11'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:42.909875 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-20e830a1-c228-4659-ad31-7eaee82032c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e90cfa6008904c62a51b9d3eab457a11'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:42.937591 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-20e830a1-c228-4659-ad31-7eaee82032c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e90cfa6008904c62a51b9d3eab457a11'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:42.939746 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:42.953248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 895/1795] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:33:42 2022] DELETE /identity/v3/users/e90cfa6008904c62a51b9d3eab457a11 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:42.954841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d58f8-af59-4b0f-9c32-a4b4d8cd58a3 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:42.955144 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d58f8-af59-4b0f-9c32-a4b4d8cd58a3 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:42.955438 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d58f8-af59-4b0f-9c32-a4b4d8cd58a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:43.029486 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5b8d5-01ed-45ab-a153-d311dd1b946a admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:43.054057 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-b50d58f8-af59-4b0f-9c32-a4b4d8cd58a3 None None] MFA Rules not processed for user `387e0ebce1cf42fca61102ce56070f9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:43.055844 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-b50d58f8-af59-4b0f-9c32-a4b4d8cd58a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:43.064573 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:43.172994 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc59be1-a14b-4a7e-8399-ea0f769e8ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:43.174454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc59be1-a14b-4a7e-8399-ea0f769e8ea1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:43.175359 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc59be1-a14b-4a7e-8399-ea0f769e8ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:43.175705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc59be1-a14b-4a7e-8399-ea0f769e8ea1 admin admin] PATH_INFO: `/v3/projects/3fe2b414d769416b834baf5c97ef6fc5/users/15ec6365d3634355b1261ed00bcbc279/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:43.182033 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc59be1-a14b-4a7e-8399-ea0f769e8ea1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fe2b414d769416b834baf5c97ef6fc5, user_id=15ec6365d3634355b1261ed00bcbc279, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:43.190981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc59be1-a14b-4a7e-8399-ea0f769e8ea1 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:43.217626 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 901/1796] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:42 2022] PUT /identity/v3/projects/3fe2b414d769416b834baf5c97ef6fc5/users/15ec6365d3634355b1261ed00bcbc279/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:43.217626 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544537b6-8edb-46fd-b1cb-f4cfcae37768 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:43.217626 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544537b6-8edb-46fd-b1cb-f4cfcae37768 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:43.217626 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544537b6-8edb-46fd-b1cb-f4cfcae37768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:43.248468 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-544537b6-8edb-46fd-b1cb-f4cfcae37768 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:43.253824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-544537b6-8edb-46fd-b1cb-f4cfcae37768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:43.266389 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 896/1797] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:42 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:43.267826 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ca8e7-c985-47cf-b6b3-4daa70b4322b None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:43.268122 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ca8e7-c985-47cf-b6b3-4daa70b4322b None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:43.270305 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ca8e7-c985-47cf-b6b3-4daa70b4322b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:43.325495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-827ca8e7-c985-47cf-b6b3-4daa70b4322b None None] MFA Rules not processed for user `15ec6365d3634355b1261ed00bcbc279`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:43.328598 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-827ca8e7-c985-47cf-b6b3-4daa70b4322b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:43.393629 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 902/1798] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:43.406561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544537b6-8edb-46fd-b1cb-f4cfcae37768 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:43.408350 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-544537b6-8edb-46fd-b1cb-f4cfcae37768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:43.440091 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 897/1799] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:43 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:43.445211 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ca8e7-c985-47cf-b6b3-4daa70b4322b None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:43.450596 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-827ca8e7-c985-47cf-b6b3-4daa70b4322b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:43.603966 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544537b6-8edb-46fd-b1cb-f4cfcae37768 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:43.616676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb16f8a-ebd6-4c38-9df5-3ae1ff11a254 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:43.617538 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb16f8a-ebd6-4c38-9df5-3ae1ff11a254 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:43.617836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb16f8a-ebd6-4c38-9df5-3ae1ff11a254 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:43.618113 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb16f8a-ebd6-4c38-9df5-3ae1ff11a254 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:43.679814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ca8e7-c985-47cf-b6b3-4daa70b4322b None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:43.710689 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a15a7a-86d7-4645-9366-3b2198eb9289 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:43.713143 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a15a7a-86d7-4645-9366-3b2198eb9289 service nova] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:43.715023 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a15a7a-86d7-4645-9366-3b2198eb9289 service nova] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:43.715546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a15a7a-86d7-4645-9366-3b2198eb9289 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:43.721982 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-82a15a7a-86d7-4645-9366-3b2198eb9289 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:43.768126 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:43.775275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb16f8a-ebd6-4c38-9df5-3ae1ff11a254 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:43.777009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb16f8a-ebd6-4c38-9df5-3ae1ff11a254 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:43.787447 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a15a7a-86d7-4645-9366-3b2198eb9289 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:43.793768 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a15a7a-86d7-4645-9366-3b2198eb9289 service nova] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:43.797458 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 903/1800] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:43 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:43.812126 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb16f8a-ebd6-4c38-9df5-3ae1ff11a254 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:43.817149 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ffb16f8a-ebd6-4c38-9df5-3ae1ff11a254 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:43.825384 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 898/1801] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:43.826285 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a15a7a-86d7-4645-9366-3b2198eb9289 service nova] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:43.839438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:43.839995 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-82a15a7a-86d7-4645-9366-3b2198eb9289 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:43.918090 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a15a7a-86d7-4645-9366-3b2198eb9289 service nova] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:43.942456 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72107ca7-b232-4b4d-87d9-c3bc2c11c61a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:43.942456 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72107ca7-b232-4b4d-87d9-c3bc2c11c61a service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:43.953162 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72107ca7-b232-4b4d-87d9-c3bc2c11c61a service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:43.953162 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72107ca7-b232-4b4d-87d9-c3bc2c11c61a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:43.999681 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb16f8a-ebd6-4c38-9df5-3ae1ff11a254 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:44.001917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72107ca7-b232-4b4d-87d9-c3bc2c11c61a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:44.006011 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176d5de-fec6-41a2-804b-449c5c7268f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:44.006902 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176d5de-fec6-41a2-804b-449c5c7268f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:44.007248 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176d5de-fec6-41a2-804b-449c5c7268f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:44.011468 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176d5de-fec6-41a2-804b-449c5c7268f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:44.012015 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72107ca7-b232-4b4d-87d9-c3bc2c11c61a service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:44.030182 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176d5de-fec6-41a2-804b-449c5c7268f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:44.031880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176d5de-fec6-41a2-804b-449c5c7268f4 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:44.063249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 899/1802] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:43 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:44.065100 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72107ca7-b232-4b4d-87d9-c3bc2c11c61a service cinder] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:44.074329 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 904/1803] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:44.107499 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72107ca7-b232-4b4d-87d9-c3bc2c11c61a service cinder] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:44.121473 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1c2c28-efaf-4361-b7f9-1cf9df0662f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:44.125223 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c2c28-efaf-4361-b7f9-1cf9df0662f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:44.125542 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c2c28-efaf-4361-b7f9-1cf9df0662f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:44.126989 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1c2c28-efaf-4361-b7f9-1cf9df0662f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:44.134648 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-5c1c2c28-efaf-4361-b7f9-1cf9df0662f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:44.168086 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176d5de-fec6-41a2-804b-449c5c7268f4 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:44.182970 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176d5de-fec6-41a2-804b-449c5c7268f4 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:44.190679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150c10a-0b94-455e-a9a5-55312df581a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:44.191279 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150c10a-0b94-455e-a9a5-55312df581a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:44.191568 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150c10a-0b94-455e-a9a5-55312df581a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:44.191852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150c10a-0b94-455e-a9a5-55312df581a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:44.201332 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-2150c10a-0b94-455e-a9a5-55312df581a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:44.307883 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1c2c28-efaf-4361-b7f9-1cf9df0662f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:44.312281 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1c2c28-efaf-4361-b7f9-1cf9df0662f5 service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:44.344816 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 900/1804] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:44 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:44.389584 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2150c10a-0b94-455e-a9a5-55312df581a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:44.404208 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2150c10a-0b94-455e-a9a5-55312df581a9 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:44.434747 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 905/1805] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:33:44.695258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c6962-b02e-42dc-b90e-29d3cfd09d30 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:44.695560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c6962-b02e-42dc-b90e-29d3cfd09d30 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:44.695853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c6962-b02e-42dc-b90e-29d3cfd09d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:44.737894 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-b98c6962-b02e-42dc-b90e-29d3cfd09d30 None None] MFA Rules not processed for user `687208d733174bffa2783e6857401f39`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:44.739357 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-b98c6962-b02e-42dc-b90e-29d3cfd09d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:44.856957 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 901/1806] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:45.151107 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150c10a-0b94-455e-a9a5-55312df581a9 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:45.154110 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-2150c10a-0b94-455e-a9a5-55312df581a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:45.177718 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150c10a-0b94-455e-a9a5-55312df581a9 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:45.205306 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44552a73-2f68-4aef-ae19-5dbf80988f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:45.206819 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44552a73-2f68-4aef-ae19-5dbf80988f9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:45.207251 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44552a73-2f68-4aef-ae19-5dbf80988f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:45.207648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44552a73-2f68-4aef-ae19-5dbf80988f9f admin admin] PATH_INFO: `/v3/projects/6c1d070c72404e148b5c158e8817cc8e` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:45.209532 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44552a73-2f68-4aef-ae19-5dbf80988f9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c1d070c72404e148b5c158e8817cc8e)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:45.214159 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44552a73-2f68-4aef-ae19-5dbf80988f9f admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:45.322935 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-44552a73-2f68-4aef-ae19-5dbf80988f9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c1d070c72404e148b5c158e8817cc8e', 'request_id': 'req-44552a73-2f68-4aef-ae19-5dbf80988f9f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:45.382803 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-44552a73-2f68-4aef-ae19-5dbf80988f9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c1d070c72404e148b5c158e8817cc8e', 'request_id': 'req-44552a73-2f68-4aef-ae19-5dbf80988f9f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:45.400763 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-44552a73-2f68-4aef-ae19-5dbf80988f9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c1d070c72404e148b5c158e8817cc8e', 'request_id': 'req-44552a73-2f68-4aef-ae19-5dbf80988f9f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:45.429050 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-44552a73-2f68-4aef-ae19-5dbf80988f9f admin admin] The token cache is being invalidate because project 6c1d070c72404e148b5c158e8817cc8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:33:45.429397 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-44552a73-2f68-4aef-ae19-5dbf80988f9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:45.430097 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-44552a73-2f68-4aef-ae19-5dbf80988f9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:45.433618 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 906/1807] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:33:45 2022] DELETE /identity/v3/projects/6c1d070c72404e148b5c158e8817cc8e => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:45.437363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c6962-b02e-42dc-b90e-29d3cfd09d30 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:45.454764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c6962-b02e-42dc-b90e-29d3cfd09d30 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:45.627842 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bcce41-1289-4fff-b36c-18b1da2abab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:45.629450 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bcce41-1289-4fff-b36c-18b1da2abab3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:45.629450 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bcce41-1289-4fff-b36c-18b1da2abab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:45.629839 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bcce41-1289-4fff-b36c-18b1da2abab3 admin admin] PATH_INFO: `/v3/users/c82bf0e3b9ec41cbb318771f12650990` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:45.633352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bcce41-1289-4fff-b36c-18b1da2abab3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c82bf0e3b9ec41cbb318771f12650990)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:45.650073 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bcce41-1289-4fff-b36c-18b1da2abab3 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:45.738070 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:45.777619 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f8bcce41-1289-4fff-b36c-18b1da2abab3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c82bf0e3b9ec41cbb318771f12650990'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:45.779446 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f8bcce41-1289-4fff-b36c-18b1da2abab3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c82bf0e3b9ec41cbb318771f12650990'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:45.797412 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f8bcce41-1289-4fff-b36c-18b1da2abab3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c82bf0e3b9ec41cbb318771f12650990'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:45.845072 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f8bcce41-1289-4fff-b36c-18b1da2abab3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c82bf0e3b9ec41cbb318771f12650990'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:45.862302 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f8bcce41-1289-4fff-b36c-18b1da2abab3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c82bf0e3b9ec41cbb318771f12650990'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:45.874180 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 902/1808] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:33:45 2022] DELETE /identity/v3/users/c82bf0e3b9ec41cbb318771f12650990 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:47.134914 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44552a73-2f68-4aef-ae19-5dbf80988f9f admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:47.136865 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-44552a73-2f68-4aef-ae19-5dbf80988f9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:47.157591 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44552a73-2f68-4aef-ae19-5dbf80988f9f admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:47.202113 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:47.307348 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3355e3-e881-49e6-ab7c-7e288977e0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:47.308448 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3355e3-e881-49e6-ab7c-7e288977e0a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:47.308883 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3355e3-e881-49e6-ab7c-7e288977e0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:47.309331 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3355e3-e881-49e6-ab7c-7e288977e0a6 admin admin] PATH_INFO: `/v3/projects/5b05cf3cf4084bd6a82d6bcbc339b69e` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:47.313226 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3355e3-e881-49e6-ab7c-7e288977e0a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b05cf3cf4084bd6a82d6bcbc339b69e)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:47.317872 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3355e3-e881-49e6-ab7c-7e288977e0a6 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:47.419577 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ad3355e3-e881-49e6-ab7c-7e288977e0a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b05cf3cf4084bd6a82d6bcbc339b69e', 'request_id': 'req-ad3355e3-e881-49e6-ab7c-7e288977e0a6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:47.427206 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ad3355e3-e881-49e6-ab7c-7e288977e0a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b05cf3cf4084bd6a82d6bcbc339b69e', 'request_id': 'req-ad3355e3-e881-49e6-ab7c-7e288977e0a6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:47.440953 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ad3355e3-e881-49e6-ab7c-7e288977e0a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b05cf3cf4084bd6a82d6bcbc339b69e', 'request_id': 'req-ad3355e3-e881-49e6-ab7c-7e288977e0a6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:47.459513 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ad3355e3-e881-49e6-ab7c-7e288977e0a6 admin admin] The token cache is being invalidate because project 5b05cf3cf4084bd6a82d6bcbc339b69e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:33:47.459967 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ad3355e3-e881-49e6-ab7c-7e288977e0a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:47.460786 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ad3355e3-e881-49e6-ab7c-7e288977e0a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:47.463119 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 907/1809] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:33:47 2022] DELETE /identity/v3/projects/5b05cf3cf4084bd6a82d6bcbc339b69e => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:47.468911 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c80d0-c111-4ea3-9ac2-6a3a055802c2 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:47.469891 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c80d0-c111-4ea3-9ac2-6a3a055802c2 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:47.470400 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c80d0-c111-4ea3-9ac2-6a3a055802c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:47.537073 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-e56c80d0-c111-4ea3-9ac2-6a3a055802c2 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:47.539265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-e56c80d0-c111-4ea3-9ac2-6a3a055802c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:47.721010 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 903/1810] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:47.729556 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f34a8-530e-4a07-9847-462e338f6844 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:47.729848 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f34a8-530e-4a07-9847-462e338f6844 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:47.730130 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f34a8-530e-4a07-9847-462e338f6844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:47.773925 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-b51f34a8-530e-4a07-9847-462e338f6844 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:47.775915 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b51f34a8-530e-4a07-9847-462e338f6844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:47.792674 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 908/1811] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:47.798603 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c80d0-c111-4ea3-9ac2-6a3a055802c2 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:47.826990 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c80d0-c111-4ea3-9ac2-6a3a055802c2 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:47.845613 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b56d40-e7f5-4433-bdc7-675924c6540f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:47.846692 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b56d40-e7f5-4433-bdc7-675924c6540f admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:47.847193 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b56d40-e7f5-4433-bdc7-675924c6540f admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:47.847692 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b56d40-e7f5-4433-bdc7-675924c6540f admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:47.848715 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b56d40-e7f5-4433-bdc7-675924c6540f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:33:47.849160 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b56d40-e7f5-4433-bdc7-675924c6540f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:47.851689 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b56d40-e7f5-4433-bdc7-675924c6540f admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:47.876603 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f34a8-530e-4a07-9847-462e338f6844 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:47.882298 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 904/1812] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:33:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:33:47.890631 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f34a8-530e-4a07-9847-462e338f6844 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:47.898047 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13edd8-8b3f-48dd-8fc6-f3764ea23ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:47.899103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13edd8-8b3f-48dd-8fc6-f3764ea23ec4 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:47.899819 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13edd8-8b3f-48dd-8fc6-f3764ea23ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:47.900485 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea13edd8-8b3f-48dd-8fc6-f3764ea23ec4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:47.901568 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea13edd8-8b3f-48dd-8fc6-f3764ea23ec4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:47.904850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea13edd8-8b3f-48dd-8fc6-f3764ea23ec4 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:47.968816 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 909/1813] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:33:47 2022] POST /identity/v3/projects => generated 383 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:47.973215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b56d40-e7f5-4433-bdc7-675924c6540f admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:47.989538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b56d40-e7f5-4433-bdc7-675924c6540f admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:48.043866 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13edd8-8b3f-48dd-8fc6-f3764ea23ec4 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:48.046040 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ea13edd8-8b3f-48dd-8fc6-f3764ea23ec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:48.195334 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea13edd8-8b3f-48dd-8fc6-f3764ea23ec4 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:48.203395 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9f4f6-9a26-4660-8bcb-a6a52c72b682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:48.203953 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9f4f6-9a26-4660-8bcb-a6a52c72b682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:48.204342 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9f4f6-9a26-4660-8bcb-a6a52c72b682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:48.204625 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9f4f6-9a26-4660-8bcb-a6a52c72b682 admin admin] PATH_INFO: `/v3/users/48f07080d7e8430d839c72895a46db55` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:48.207025 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc9f4f6-9a26-4660-8bcb-a6a52c72b682 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48f07080d7e8430d839c72895a46db55)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:48.210612 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc9f4f6-9a26-4660-8bcb-a6a52c72b682 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:48.264776 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e62195-6129-4e19-86d8-e25764d42004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:48.265067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:48.265178 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e62195-6129-4e19-86d8-e25764d42004 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:48.265626 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e62195-6129-4e19-86d8-e25764d42004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:48.267532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e62195-6129-4e19-86d8-e25764d42004 admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:48.267532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e62195-6129-4e19-86d8-e25764d42004 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:48.273774 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e62195-6129-4e19-86d8-e25764d42004 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:48.296841 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-efc9f4f6-9a26-4660-8bcb-a6a52c72b682 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48f07080d7e8430d839c72895a46db55'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:48.334448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 905/1814] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:33:47 2022] POST /identity/v3/users => generated 292 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:48.344539 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-efc9f4f6-9a26-4660-8bcb-a6a52c72b682 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48f07080d7e8430d839c72895a46db55'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:48.355369 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e62195-6129-4e19-86d8-e25764d42004 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:48.371417 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-efc9f4f6-9a26-4660-8bcb-a6a52c72b682 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48f07080d7e8430d839c72895a46db55'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:48.372158 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-efc9f4f6-9a26-4660-8bcb-a6a52c72b682 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48f07080d7e8430d839c72895a46db55'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:48.375362 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-efc9f4f6-9a26-4660-8bcb-a6a52c72b682 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48f07080d7e8430d839c72895a46db55'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:48.376208 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e62195-6129-4e19-86d8-e25764d42004 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:48.390564 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f7381-acd6-407b-8a51-4b0da19696fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:48.391822 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f7381-acd6-407b-8a51-4b0da19696fe admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:48.392479 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f7381-acd6-407b-8a51-4b0da19696fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:48.394200 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f7381-acd6-407b-8a51-4b0da19696fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:48.407230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f7381-acd6-407b-8a51-4b0da19696fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:48.409336 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f7381-acd6-407b-8a51-4b0da19696fe admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:48.418574 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 910/1815] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:33:48 2022] DELETE /identity/v3/users/48f07080d7e8430d839c72895a46db55 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:48.421358 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10112e00-b635-4de0-b4ab-66852e1ec86f None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:48.421657 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10112e00-b635-4de0-b4ab-66852e1ec86f None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:48.421942 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10112e00-b635-4de0-b4ab-66852e1ec86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:48.441154 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 906/1816] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:48.445178 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f7381-acd6-407b-8a51-4b0da19696fe admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:48.449195 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-3c1f7381-acd6-407b-8a51-4b0da19696fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:48.455375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-10112e00-b635-4de0-b4ab-66852e1ec86f None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:48.456764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-10112e00-b635-4de0-b4ab-66852e1ec86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:48.473974 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f7381-acd6-407b-8a51-4b0da19696fe admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:48.482709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:48.567670 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 911/1817] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:33:48.602384 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c820a-b325-4f06-94c6-eed290e65a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:48.603284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c820a-b325-4f06-94c6-eed290e65a31 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:48.603704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c820a-b325-4f06-94c6-eed290e65a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:48.604089 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c820a-b325-4f06-94c6-eed290e65a31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:48.604923 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4c820a-b325-4f06-94c6-eed290e65a31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:48.609201 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4c820a-b325-4f06-94c6-eed290e65a31 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:48.610685 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10112e00-b635-4de0-b4ab-66852e1ec86f None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:48.612707 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-10112e00-b635-4de0-b4ab-66852e1ec86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:48.631378 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 907/1818] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:48.636836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10112e00-b635-4de0-b4ab-66852e1ec86f None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:48.637965 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c820a-b325-4f06-94c6-eed290e65a31 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:48.653205 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ff54b-fc6d-4053-b8d5-b9d0d9b68602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:48.654128 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ff54b-fc6d-4053-b8d5-b9d0d9b68602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:48.655115 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ff54b-fc6d-4053-b8d5-b9d0d9b68602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:48.655115 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ff54b-fc6d-4053-b8d5-b9d0d9b68602 admin admin] PATH_INFO: `/v3/users/4314899eb1804171923230964700dfb6` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:48.658799 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ff54b-fc6d-4053-b8d5-b9d0d9b68602 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4314899eb1804171923230964700dfb6)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:48.661593 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ff54b-fc6d-4053-b8d5-b9d0d9b68602 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:48.678967 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c820a-b325-4f06-94c6-eed290e65a31 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:48.697593 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8b2ab-1c67-46fa-bfb5-1e81e4679f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:48.698508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8b2ab-1c67-46fa-bfb5-1e81e4679f97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:48.698801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8b2ab-1c67-46fa-bfb5-1e81e4679f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:48.699097 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde8b2ab-1c67-46fa-bfb5-1e81e4679f97 admin admin] PATH_INFO: `/v3/projects/4116bb505734419888a2f3c40d29406b/users/3cbb4314dbe6468ca0b8c72183944759/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:48.750689 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8b2ab-1c67-46fa-bfb5-1e81e4679f97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4116bb505734419888a2f3c40d29406b, user_id=3cbb4314dbe6468ca0b8c72183944759, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:48.757556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde8b2ab-1c67-46fa-bfb5-1e81e4679f97 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:48.795497 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 908/1819] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:48 2022] PUT /identity/v3/projects/4116bb505734419888a2f3c40d29406b/users/3cbb4314dbe6468ca0b8c72183944759/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:48.796971 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8b2ab-1c67-46fa-bfb5-1e81e4679f97 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:48.799108 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-dde8b2ab-1c67-46fa-bfb5-1e81e4679f97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:48.831775 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-af7ff54b-fc6d-4053-b8d5-b9d0d9b68602 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4314899eb1804171923230964700dfb6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:48.873507 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-af7ff54b-fc6d-4053-b8d5-b9d0d9b68602 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4314899eb1804171923230964700dfb6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:48.887033 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-af7ff54b-fc6d-4053-b8d5-b9d0d9b68602 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4314899eb1804171923230964700dfb6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:48.887898 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-af7ff54b-fc6d-4053-b8d5-b9d0d9b68602 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4314899eb1804171923230964700dfb6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:48.888882 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-af7ff54b-fc6d-4053-b8d5-b9d0d9b68602 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4314899eb1804171923230964700dfb6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:48.907634 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 912/1820] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:33:48 2022] DELETE /identity/v3/users/4314899eb1804171923230964700dfb6 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:48.907634 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc19c49-1ec4-432b-8736-4e597c555548 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:48.907634 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc19c49-1ec4-432b-8736-4e597c555548 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:48.907634 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc19c49-1ec4-432b-8736-4e597c555548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:48.939799 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-4cc19c49-1ec4-432b-8736-4e597c555548 None None] MFA Rules not processed for user `3cbb4314dbe6468ca0b8c72183944759`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:48.941381 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-4cc19c49-1ec4-432b-8736-4e597c555548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:48.969229 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:49.032270 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde8b2ab-1c67-46fa-bfb5-1e81e4679f97 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:49.065126 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 913/1821] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:49.069431 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51e989-d91d-4e88-b7f8-c2c06c0230d5 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:49.070637 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51e989-d91d-4e88-b7f8-c2c06c0230d5 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:49.072530 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51e989-d91d-4e88-b7f8-c2c06c0230d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:49.123253 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-1e51e989-d91d-4e88-b7f8-c2c06c0230d5 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:49.125349 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1e51e989-d91d-4e88-b7f8-c2c06c0230d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:49.229173 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc84aa-29f0-42c6-997e-baf942fac7af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:49.229868 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc84aa-29f0-42c6-997e-baf942fac7af service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:49.230158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc84aa-29f0-42c6-997e-baf942fac7af service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:49.230440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc84aa-29f0-42c6-997e-baf942fac7af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:49.239327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-b7bc84aa-29f0-42c6-997e-baf942fac7af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:49.248192 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 914/1822] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:49.248192 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51e989-d91d-4e88-b7f8-c2c06c0230d5 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:49.260708 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1e51e989-d91d-4e88-b7f8-c2c06c0230d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:49.280031 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc84aa-29f0-42c6-997e-baf942fac7af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:49.281684 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc84aa-29f0-42c6-997e-baf942fac7af service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:49.307427 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 909/1823] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:49.309502 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc84aa-29f0-42c6-997e-baf942fac7af service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:49.309946 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51e989-d91d-4e88-b7f8-c2c06c0230d5 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:49.325764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-b7bc84aa-29f0-42c6-997e-baf942fac7af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:49.337677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4e7fa-bbbc-44e3-90db-953112970acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:49.341523 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4e7fa-bbbc-44e3-90db-953112970acc admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:49.341856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4e7fa-bbbc-44e3-90db-953112970acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:49.342275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4e7fa-bbbc-44e3-90db-953112970acc admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:49.343139 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a4e7fa-bbbc-44e3-90db-953112970acc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:49.349571 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a4e7fa-bbbc-44e3-90db-953112970acc admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:49.358545 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc84aa-29f0-42c6-997e-baf942fac7af service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:49.399773 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a36e2-a0ca-466e-9065-dee052eef6b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:49.400501 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a36e2-a0ca-466e-9065-dee052eef6b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:49.400844 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a36e2-a0ca-466e-9065-dee052eef6b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:49.401121 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a36e2-a0ca-466e-9065-dee052eef6b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:49.435871 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a36e2-a0ca-466e-9065-dee052eef6b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:49.441268 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a36e2-a0ca-466e-9065-dee052eef6b8 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:49.465330 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 915/1824] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:33:49 2022] POST /identity/v3/projects => generated 381 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:49.469745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4e7fa-bbbc-44e3-90db-953112970acc admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:49.479349 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 910/1825] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:49.651357 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4e7fa-bbbc-44e3-90db-953112970acc admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:49.658810 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b349f34-1b88-4dcc-b04a-d89006537fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:49.660277 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b349f34-1b88-4dcc-b04a-d89006537fb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:49.660949 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b349f34-1b88-4dcc-b04a-d89006537fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:49.661379 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b349f34-1b88-4dcc-b04a-d89006537fb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:49.662353 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b349f34-1b88-4dcc-b04a-d89006537fb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:49.681286 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b349f34-1b88-4dcc-b04a-d89006537fb5 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:49.726532 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 916/1826] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:33:49 2022] POST /identity/v3/users => generated 297 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:49.731457 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a36e2-a0ca-466e-9065-dee052eef6b8 service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:49.755567 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a36e2-a0ca-466e-9065-dee052eef6b8 service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:49.767218 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917049aa-5e61-416a-8052-b603610c3a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:49.768374 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917049aa-5e61-416a-8052-b603610c3a16 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:49.769578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917049aa-5e61-416a-8052-b603610c3a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:49.769871 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917049aa-5e61-416a-8052-b603610c3a16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:49.770635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917049aa-5e61-416a-8052-b603610c3a16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:49.772455 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917049aa-5e61-416a-8052-b603610c3a16 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:49.813893 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 911/1827] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:49.817907 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b349f34-1b88-4dcc-b04a-d89006537fb5 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:49.844229 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b349f34-1b88-4dcc-b04a-d89006537fb5 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:49.859193 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da316f8-2823-4350-b442-892a34acd1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:49.862720 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da316f8-2823-4350-b442-892a34acd1cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:49.863022 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da316f8-2823-4350-b442-892a34acd1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:49.866593 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da316f8-2823-4350-b442-892a34acd1cb admin admin] PATH_INFO: `/v3/projects/18333e2a50f94a6ab0e819609de48543/users/ea065ecf98b941beb1b7a2ec68eb17da/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:49.925176 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da316f8-2823-4350-b442-892a34acd1cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=18333e2a50f94a6ab0e819609de48543, user_id=ea065ecf98b941beb1b7a2ec68eb17da, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:49.928048 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da316f8-2823-4350-b442-892a34acd1cb admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:49.936202 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:49.954373 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 917/1828] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:49 2022] PUT /identity/v3/projects/18333e2a50f94a6ab0e819609de48543/users/ea065ecf98b941beb1b7a2ec68eb17da/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:49.959049 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917049aa-5e61-416a-8052-b603610c3a16 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:49.972056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917049aa-5e61-416a-8052-b603610c3a16 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:49.989956 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:50.117086 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a30f41-34f0-4a7c-9d00-b244c0b23c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:50.117926 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a30f41-34f0-4a7c-9d00-b244c0b23c17 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:50.118854 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a30f41-34f0-4a7c-9d00-b244c0b23c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:50.119222 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a30f41-34f0-4a7c-9d00-b244c0b23c17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:50.120712 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a30f41-34f0-4a7c-9d00-b244c0b23c17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:50.125840 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a30f41-34f0-4a7c-9d00-b244c0b23c17 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:50.144042 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 912/1829] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:33:50.151681 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da316f8-2823-4350-b442-892a34acd1cb admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:50.167899 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da316f8-2823-4350-b442-892a34acd1cb admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:50.188768 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9541c66-7a4c-4aad-802f-b4bc966d094a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:50.189715 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9541c66-7a4c-4aad-802f-b4bc966d094a admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:50.190007 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9541c66-7a4c-4aad-802f-b4bc966d094a admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:50.190301 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9541c66-7a4c-4aad-802f-b4bc966d094a admin admin] PATH_INFO: `/v3/projects/18333e2a50f94a6ab0e819609de48543/users/ea065ecf98b941beb1b7a2ec68eb17da/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:50.197883 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9541c66-7a4c-4aad-802f-b4bc966d094a admin admin] RBAC: Authorizing `identity:create_grant(project_id=18333e2a50f94a6ab0e819609de48543, user_id=ea065ecf98b941beb1b7a2ec68eb17da, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:50.204198 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9541c66-7a4c-4aad-802f-b4bc966d094a admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:50.233141 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 918/1830] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:50 2022] PUT /identity/v3/projects/18333e2a50f94a6ab0e819609de48543/users/ea065ecf98b941beb1b7a2ec68eb17da/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:50.244934 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309176c7-3731-4c68-a822-cb084f9500a0 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:50.245261 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309176c7-3731-4c68-a822-cb084f9500a0 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:50.245481 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309176c7-3731-4c68-a822-cb084f9500a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:50.293033 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-309176c7-3731-4c68-a822-cb084f9500a0 None None] MFA Rules not processed for user `ea065ecf98b941beb1b7a2ec68eb17da`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:50.295136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-309176c7-3731-4c68-a822-cb084f9500a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:50.437970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 913/1831] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:50.447636 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9541c66-7a4c-4aad-802f-b4bc966d094a admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:50.450361 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e9541c66-7a4c-4aad-802f-b4bc966d094a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:50.511737 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc6786-c200-4636-a254-1362af498fcc None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:50.512327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc6786-c200-4636-a254-1362af498fcc None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:50.513046 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc6786-c200-4636-a254-1362af498fcc None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:50.514503 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 914/1832] 10.222.0.16 () {54 vars in 921 bytes} [Mon May 23 12:33:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:33:50.554768 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9541c66-7a4c-4aad-802f-b4bc966d094a admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:50.560650 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0518448-2f0f-462d-b26f-5ee48087aebd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:50.561426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0518448-2f0f-462d-b26f-5ee48087aebd service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:50.561619 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0518448-2f0f-462d-b26f-5ee48087aebd service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:50.561913 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0518448-2f0f-462d-b26f-5ee48087aebd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:50.566915 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5e241-5292-4af0-bc5c-9e66db142bba None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:50.567424 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5e241-5292-4af0-bc5c-9e66db142bba None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:50.569083 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5e241-5292-4af0-bc5c-9e66db142bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:50.584239 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0518448-2f0f-462d-b26f-5ee48087aebd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:50.586102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0518448-2f0f-462d-b26f-5ee48087aebd service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:50.592776 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-beb5e241-5292-4af0-bc5c-9e66db142bba None None] MFA Rules not processed for user `b83535b443a84275a160cd84cb3ad5a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:50.594591 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-beb5e241-5292-4af0-bc5c-9e66db142bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:50.644810 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 919/1833] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:50 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:50.719105 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 915/1834] 10.222.0.16 () {58 vars in 1018 bytes} [Mon May 23 12:33:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:33:50.756779 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0518448-2f0f-462d-b26f-5ee48087aebd service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:50.758522 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e0518448-2f0f-462d-b26f-5ee48087aebd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:50.759404 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb5e241-5292-4af0-bc5c-9e66db142bba None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:50.770656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb5e241-5292-4af0-bc5c-9e66db142bba None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:50.775653 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0964e3c-70cb-49d9-bcce-0ccd7029f0d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:50.776486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0964e3c-70cb-49d9-bcce-0ccd7029f0d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:50.776778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0964e3c-70cb-49d9-bcce-0ccd7029f0d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:50.777062 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0964e3c-70cb-49d9-bcce-0ccd7029f0d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:50.777684 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0518448-2f0f-462d-b26f-5ee48087aebd service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:50.785683 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe08dd0-60e6-4abd-b133-d4ea4f88f92e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:50.786361 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe08dd0-60e6-4abd-b133-d4ea4f88f92e service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:50.786649 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe08dd0-60e6-4abd-b133-d4ea4f88f92e service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:50.786930 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe08dd0-60e6-4abd-b133-d4ea4f88f92e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:50.798159 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe08dd0-60e6-4abd-b133-d4ea4f88f92e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:50.799765 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe08dd0-60e6-4abd-b133-d4ea4f88f92e service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:50.812554 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:50.817732 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 920/1835] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:50.863035 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0964e3c-70cb-49d9-bcce-0ccd7029f0d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:50.864586 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0964e3c-70cb-49d9-bcce-0ccd7029f0d5 service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:50.890567 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 916/1836] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:33:51.062131 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe08dd0-60e6-4abd-b133-d4ea4f88f92e service nova] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:51.063413 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-fbe08dd0-60e6-4abd-b133-d4ea4f88f92e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:51.187700 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe08dd0-60e6-4abd-b133-d4ea4f88f92e service nova] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:51.193739 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b76168-ac97-4e1e-8e12-69c8d1b8ca77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:51.194632 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b76168-ac97-4e1e-8e12-69c8d1b8ca77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:51.195302 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b76168-ac97-4e1e-8e12-69c8d1b8ca77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:51.196122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b76168-ac97-4e1e-8e12-69c8d1b8ca77 admin admin] PATH_INFO: `/v3/projects/be5dc248c0764095a62067552b30f6d4` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:51.197729 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b76168-ac97-4e1e-8e12-69c8d1b8ca77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be5dc248c0764095a62067552b30f6d4)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:51.199900 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b76168-ac97-4e1e-8e12-69c8d1b8ca77 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:51.283475 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d6b76168-ac97-4e1e-8e12-69c8d1b8ca77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be5dc248c0764095a62067552b30f6d4', 'request_id': 'req-d6b76168-ac97-4e1e-8e12-69c8d1b8ca77'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:51.297060 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d6b76168-ac97-4e1e-8e12-69c8d1b8ca77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be5dc248c0764095a62067552b30f6d4', 'request_id': 'req-d6b76168-ac97-4e1e-8e12-69c8d1b8ca77'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:51.308962 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d6b76168-ac97-4e1e-8e12-69c8d1b8ca77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be5dc248c0764095a62067552b30f6d4', 'request_id': 'req-d6b76168-ac97-4e1e-8e12-69c8d1b8ca77'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:51.348652 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d6b76168-ac97-4e1e-8e12-69c8d1b8ca77 admin admin] The token cache is being invalidate because project be5dc248c0764095a62067552b30f6d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:33:51.349608 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d6b76168-ac97-4e1e-8e12-69c8d1b8ca77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:51.352015 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d6b76168-ac97-4e1e-8e12-69c8d1b8ca77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:51.356488 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 921/1837] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:33:51 2022] DELETE /identity/v3/projects/be5dc248c0764095a62067552b30f6d4 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:51.361671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0964e3c-70cb-49d9-bcce-0ccd7029f0d5 service cinder] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:51.383903 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b76168-ac97-4e1e-8e12-69c8d1b8ca77 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:51.386318 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d6b76168-ac97-4e1e-8e12-69c8d1b8ca77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:51.388326 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0964e3c-70cb-49d9-bcce-0ccd7029f0d5 service cinder] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:51.399585 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:51.497535 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1c28ff-ccac-484b-8130-b0426064667e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:51.499532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c28ff-ccac-484b-8130-b0426064667e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:51.500345 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c28ff-ccac-484b-8130-b0426064667e admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:51.500770 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c28ff-ccac-484b-8130-b0426064667e admin admin] PATH_INFO: `/v3/users/387e0ebce1cf42fca61102ce56070f9b` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:51.503027 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1c28ff-ccac-484b-8130-b0426064667e admin admin] RBAC: Authorizing `identity:delete_user(user_id=387e0ebce1cf42fca61102ce56070f9b)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:51.506397 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1c28ff-ccac-484b-8130-b0426064667e admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:51.547698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b76168-ac97-4e1e-8e12-69c8d1b8ca77 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:51.565626 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808f367-c217-4ae5-ace0-aa274685c414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:51.567035 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808f367-c217-4ae5-ace0-aa274685c414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:51.567346 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808f367-c217-4ae5-ace0-aa274685c414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:51.567837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808f367-c217-4ae5-ace0-aa274685c414 admin admin] PATH_INFO: `/v3/users/3cbb4314dbe6468ca0b8c72183944759` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:51.569482 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808f367-c217-4ae5-ace0-aa274685c414 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cbb4314dbe6468ca0b8c72183944759)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:51.571822 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808f367-c217-4ae5-ace0-aa274685c414 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:51.657782 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-8a1c28ff-ccac-484b-8130-b0426064667e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '387e0ebce1cf42fca61102ce56070f9b'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:51.658503 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-8a1c28ff-ccac-484b-8130-b0426064667e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '387e0ebce1cf42fca61102ce56070f9b'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:51.673853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a808f367-c217-4ae5-ace0-aa274685c414 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cbb4314dbe6468ca0b8c72183944759'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:51.688965 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-8a1c28ff-ccac-484b-8130-b0426064667e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '387e0ebce1cf42fca61102ce56070f9b'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:51.717559 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a808f367-c217-4ae5-ace0-aa274685c414 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cbb4314dbe6468ca0b8c72183944759'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:51.721630 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-8a1c28ff-ccac-484b-8130-b0426064667e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '387e0ebce1cf42fca61102ce56070f9b'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:51.731510 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a808f367-c217-4ae5-ace0-aa274685c414 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cbb4314dbe6468ca0b8c72183944759'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:51.734570 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a808f367-c217-4ae5-ace0-aa274685c414 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cbb4314dbe6468ca0b8c72183944759'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:51.735817 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a808f367-c217-4ae5-ace0-aa274685c414 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cbb4314dbe6468ca0b8c72183944759'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:51.738249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-8a1c28ff-ccac-484b-8130-b0426064667e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '387e0ebce1cf42fca61102ce56070f9b'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:51.746468 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 917/1838] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:33:51 2022] DELETE /identity/v3/users/387e0ebce1cf42fca61102ce56070f9b => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:51.750456 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1c28ff-ccac-484b-8130-b0426064667e admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:51.757129 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-8a1c28ff-ccac-484b-8130-b0426064667e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:51.766146 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 922/1839] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:33:51 2022] DELETE /identity/v3/users/3cbb4314dbe6468ca0b8c72183944759 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:51.771177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485210fc-a36d-4c04-a906-9228fddfc469 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:51.771420 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485210fc-a36d-4c04-a906-9228fddfc469 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:51.771823 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485210fc-a36d-4c04-a906-9228fddfc469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:51.800779 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-485210fc-a36d-4c04-a906-9228fddfc469 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:51.802110 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-485210fc-a36d-4c04-a906-9228fddfc469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:51.880783 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:51.910279 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1c28ff-ccac-484b-8130-b0426064667e admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:52.007240 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 923/1840] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:52.064272 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485210fc-a36d-4c04-a906-9228fddfc469 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:52.065708 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-485210fc-a36d-4c04-a906-9228fddfc469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:52.077031 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8cbef-4013-44b6-bcd2-1cd787371e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:52.077927 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8cbef-4013-44b6-bcd2-1cd787371e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:52.078211 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8cbef-4013-44b6-bcd2-1cd787371e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:52.078486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8cbef-4013-44b6-bcd2-1cd787371e97 admin admin] PATH_INFO: `/v3/projects/00dc00f3b98e4ca6878b1a89c7eda087` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:52.082316 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8cbef-4013-44b6-bcd2-1cd787371e97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00dc00f3b98e4ca6878b1a89c7eda087)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:52.084494 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8cbef-4013-44b6-bcd2-1cd787371e97 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:52.116370 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:52.177340 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485210fc-a36d-4c04-a906-9228fddfc469 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:52.189316 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-3db8cbef-4013-44b6-bcd2-1cd787371e97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00dc00f3b98e4ca6878b1a89c7eda087', 'request_id': 'req-3db8cbef-4013-44b6-bcd2-1cd787371e97'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:52.203009 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-3db8cbef-4013-44b6-bcd2-1cd787371e97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00dc00f3b98e4ca6878b1a89c7eda087', 'request_id': 'req-3db8cbef-4013-44b6-bcd2-1cd787371e97'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:52.212865 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-3db8cbef-4013-44b6-bcd2-1cd787371e97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00dc00f3b98e4ca6878b1a89c7eda087', 'request_id': 'req-3db8cbef-4013-44b6-bcd2-1cd787371e97'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:52.227580 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-3db8cbef-4013-44b6-bcd2-1cd787371e97 admin admin] The token cache is being invalidate because project 00dc00f3b98e4ca6878b1a89c7eda087 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:33:52.227869 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-3db8cbef-4013-44b6-bcd2-1cd787371e97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:52.228829 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-3db8cbef-4013-44b6-bcd2-1cd787371e97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:52.230446 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 918/1841] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:33:51 2022] DELETE /identity/v3/projects/00dc00f3b98e4ca6878b1a89c7eda087 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:52.235492 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8cbef-4013-44b6-bcd2-1cd787371e97 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:52.247386 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-3db8cbef-4013-44b6-bcd2-1cd787371e97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:52.260831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc00c97f-74a1-4f28-b008-23dff3602aeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:52.261511 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00c97f-74a1-4f28-b008-23dff3602aeb service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:52.261790 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00c97f-74a1-4f28-b008-23dff3602aeb service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:52.262073 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00c97f-74a1-4f28-b008-23dff3602aeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:52.265158 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-fc00c97f-74a1-4f28-b008-23dff3602aeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:52.339308 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8cbef-4013-44b6-bcd2-1cd787371e97 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:52.344517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651d319-aab6-48ab-ac05-b9179e4827ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:52.345571 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651d319-aab6-48ab-ac05-b9179e4827ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:52.345859 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651d319-aab6-48ab-ac05-b9179e4827ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:52.346134 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651d319-aab6-48ab-ac05-b9179e4827ec admin admin] PATH_INFO: `/v3/users/6474b34465a94ce7b5898123d055ccab` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:52.347426 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651d319-aab6-48ab-ac05-b9179e4827ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=6474b34465a94ce7b5898123d055ccab)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:52.349612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651d319-aab6-48ab-ac05-b9179e4827ec admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:52.389036 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc00c97f-74a1-4f28-b008-23dff3602aeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:52.390830 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc00c97f-74a1-4f28-b008-23dff3602aeb service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:52.400917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:52.408287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 924/1842] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:52.431806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-d651d319-aab6-48ab-ac05-b9179e4827ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6474b34465a94ce7b5898123d055ccab'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:52.433101 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-d651d319-aab6-48ab-ac05-b9179e4827ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6474b34465a94ce7b5898123d055ccab'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:52.443724 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-d651d319-aab6-48ab-ac05-b9179e4827ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6474b34465a94ce7b5898123d055ccab'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:52.473054 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-d651d319-aab6-48ab-ac05-b9179e4827ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6474b34465a94ce7b5898123d055ccab'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:52.486704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-d651d319-aab6-48ab-ac05-b9179e4827ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6474b34465a94ce7b5898123d055ccab'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:52.504347 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 919/1843] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:33:52 2022] DELETE /identity/v3/users/6474b34465a94ce7b5898123d055ccab => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:33:52.941579 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc00c97f-74a1-4f28-b008-23dff3602aeb service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:52.945410 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-fc00c97f-74a1-4f28-b008-23dff3602aeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:52.973664 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc00c97f-74a1-4f28-b008-23dff3602aeb service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:53.121929 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd1680-e798-45c8-81e0-db10c37f79f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:53.122886 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd1680-e798-45c8-81e0-db10c37f79f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:53.123324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd1680-e798-45c8-81e0-db10c37f79f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:53.123617 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd1680-e798-45c8-81e0-db10c37f79f5 admin admin] PATH_INFO: `/v3/projects/0a9fe3c7c3c94f008ac7788c42894376` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:53.126517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd1680-e798-45c8-81e0-db10c37f79f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a9fe3c7c3c94f008ac7788c42894376)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:53.128620 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd1680-e798-45c8-81e0-db10c37f79f5 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:53.183253 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:53.202622 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c7bd1680-e798-45c8-81e0-db10c37f79f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a9fe3c7c3c94f008ac7788c42894376', 'request_id': 'req-c7bd1680-e798-45c8-81e0-db10c37f79f5'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:53.216153 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c7bd1680-e798-45c8-81e0-db10c37f79f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a9fe3c7c3c94f008ac7788c42894376', 'request_id': 'req-c7bd1680-e798-45c8-81e0-db10c37f79f5'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:53.226971 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c7bd1680-e798-45c8-81e0-db10c37f79f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a9fe3c7c3c94f008ac7788c42894376', 'request_id': 'req-c7bd1680-e798-45c8-81e0-db10c37f79f5'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:53.261176 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c7bd1680-e798-45c8-81e0-db10c37f79f5 admin admin] The token cache is being invalidate because project 0a9fe3c7c3c94f008ac7788c42894376 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:33:53.262289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c7bd1680-e798-45c8-81e0-db10c37f79f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:53.262289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c7bd1680-e798-45c8-81e0-db10c37f79f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:53.264441 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 925/1844] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:33:52 2022] DELETE /identity/v3/projects/0a9fe3c7c3c94f008ac7788c42894376 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:53.270670 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bdceb-30c0-45ef-b864-974ba86696eb None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:53.271431 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bdceb-30c0-45ef-b864-974ba86696eb None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:53.271431 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41bdceb-30c0-45ef-b864-974ba86696eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:53.299326 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-d41bdceb-30c0-45ef-b864-974ba86696eb None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:53.300774 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-d41bdceb-30c0-45ef-b864-974ba86696eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:53.413674 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 920/1845] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:53.421486 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448d903-61aa-42a6-bc72-586cb26cb2c3 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:53.421804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448d903-61aa-42a6-bc72-586cb26cb2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:53.422042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9448d903-61aa-42a6-bc72-586cb26cb2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:53.466240 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-9448d903-61aa-42a6-bc72-586cb26cb2c3 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:53.468935 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-9448d903-61aa-42a6-bc72-586cb26cb2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:53.479837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 926/1846] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:53.486830 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41bdceb-30c0-45ef-b864-974ba86696eb None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:53.504987 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41bdceb-30c0-45ef-b864-974ba86696eb None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:53.513876 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062d6c84-04bf-4d3e-a5b5-bfd675a65bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:53.514694 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d6c84-04bf-4d3e-a5b5-bfd675a65bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:53.515127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d6c84-04bf-4d3e-a5b5-bfd675a65bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:53.515613 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d6c84-04bf-4d3e-a5b5-bfd675a65bf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:53.516580 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062d6c84-04bf-4d3e-a5b5-bfd675a65bf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:33:53.517496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062d6c84-04bf-4d3e-a5b5-bfd675a65bf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:53.519688 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062d6c84-04bf-4d3e-a5b5-bfd675a65bf6 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:53.547587 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 921/1847] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:33:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:33:53.552912 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448d903-61aa-42a6-bc72-586cb26cb2c3 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:53.568394 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9448d903-61aa-42a6-bc72-586cb26cb2c3 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:53.576478 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927c209-23ca-4ebd-9939-ea95589ff6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:53.577290 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927c209-23ca-4ebd-9939-ea95589ff6df admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:53.577729 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927c209-23ca-4ebd-9939-ea95589ff6df admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:53.578231 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927c209-23ca-4ebd-9939-ea95589ff6df admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:53.579142 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6927c209-23ca-4ebd-9939-ea95589ff6df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:53.582016 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6927c209-23ca-4ebd-9939-ea95589ff6df admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:53.657099 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 927/1848] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:33:53 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:53.659195 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062d6c84-04bf-4d3e-a5b5-bfd675a65bf6 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:53.673797 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062d6c84-04bf-4d3e-a5b5-bfd675a65bf6 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:53.777334 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a714d-b0bb-4546-a618-3051067a2fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:53.778311 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a714d-b0bb-4546-a618-3051067a2fba admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:53.778779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a714d-b0bb-4546-a618-3051067a2fba admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:53.779211 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a714d-b0bb-4546-a618-3051067a2fba admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:53.780119 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a714d-b0bb-4546-a618-3051067a2fba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:53.782467 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a714d-b0bb-4546-a618-3051067a2fba admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:53.833472 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 922/1849] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:33:53 2022] POST /identity/v3/users => generated 294 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:33:53.838901 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927c209-23ca-4ebd-9939-ea95589ff6df admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:53.861601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927c209-23ca-4ebd-9939-ea95589ff6df admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:53.893508 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124af232-ff2b-4f85-acf9-13cdcdf66872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:53.895398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124af232-ff2b-4f85-acf9-13cdcdf66872 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:53.895746 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124af232-ff2b-4f85-acf9-13cdcdf66872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:53.896728 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124af232-ff2b-4f85-acf9-13cdcdf66872 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:53.898776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124af232-ff2b-4f85-acf9-13cdcdf66872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:53.904823 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124af232-ff2b-4f85-acf9-13cdcdf66872 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:53.935787 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 928/1850] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:53.937495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a714d-b0bb-4546-a618-3051067a2fba admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:53.949370 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124af232-ff2b-4f85-acf9-13cdcdf66872 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:53.954206 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a714d-b0bb-4546-a618-3051067a2fba admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:53.955727 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-124af232-ff2b-4f85-acf9-13cdcdf66872 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:53.969206 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944d31a-9bce-47b0-b924-9de184d72002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:53.969925 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944d31a-9bce-47b0-b924-9de184d72002 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:53.970249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944d31a-9bce-47b0-b924-9de184d72002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:53.970538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4944d31a-9bce-47b0-b924-9de184d72002 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:53.971298 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4944d31a-9bce-47b0-b924-9de184d72002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:53.976109 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4944d31a-9bce-47b0-b924-9de184d72002 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:53.988258 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124af232-ff2b-4f85-acf9-13cdcdf66872 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:53.995529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 923/1851] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:54.003107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944d31a-9bce-47b0-b924-9de184d72002 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:54.004202 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3766f-a1a5-4074-9258-d9cead6af13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:54.005134 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3766f-a1a5-4074-9258-d9cead6af13c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:54.005517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3766f-a1a5-4074-9258-d9cead6af13c admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:54.005867 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3766f-a1a5-4074-9258-d9cead6af13c admin admin] PATH_INFO: `/v3/projects/76eabfbdf6b546a383e8619acd9e0881` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:54.007768 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3766f-a1a5-4074-9258-d9cead6af13c admin admin] RBAC: Authorizing `identity:delete_project(project_id=76eabfbdf6b546a383e8619acd9e0881)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:54.034149 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3766f-a1a5-4074-9258-d9cead6af13c admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:54.042200 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4944d31a-9bce-47b0-b924-9de184d72002 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:54.054933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67df9972-fe5c-4bc1-9887-faa459623e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:54.055311 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df9972-fe5c-4bc1-9887-faa459623e65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:54.056159 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df9972-fe5c-4bc1-9887-faa459623e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:54.056467 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df9972-fe5c-4bc1-9887-faa459623e65 admin admin] PATH_INFO: `/v3/projects/9aa39311ac3c442bbe55c6f32d0b0680/users/690fcaa67b5e469a8f32a748f837fada/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:54.084282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:54.110812 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df9972-fe5c-4bc1-9887-faa459623e65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9aa39311ac3c442bbe55c6f32d0b0680, user_id=690fcaa67b5e469a8f32a748f837fada, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:54.121752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67df9972-fe5c-4bc1-9887-faa459623e65 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:54.147900 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 924/1852] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:53 2022] PUT /identity/v3/projects/9aa39311ac3c442bbe55c6f32d0b0680/users/690fcaa67b5e469a8f32a748f837fada/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:54.157612 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5da3766f-a1a5-4074-9258-d9cead6af13c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76eabfbdf6b546a383e8619acd9e0881', 'request_id': 'req-5da3766f-a1a5-4074-9258-d9cead6af13c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:54.159137 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a86cc4-ae12-4c42-be9f-f27b25140442 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:54.159504 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a86cc4-ae12-4c42-be9f-f27b25140442 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:54.159856 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a86cc4-ae12-4c42-be9f-f27b25140442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:54.191445 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5da3766f-a1a5-4074-9258-d9cead6af13c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76eabfbdf6b546a383e8619acd9e0881', 'request_id': 'req-5da3766f-a1a5-4074-9258-d9cead6af13c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:54.197725 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-a6a86cc4-ae12-4c42-be9f-f27b25140442 None None] MFA Rules not processed for user `690fcaa67b5e469a8f32a748f837fada`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:54.199087 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-a6a86cc4-ae12-4c42-be9f-f27b25140442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:54.200661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5da3766f-a1a5-4074-9258-d9cead6af13c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76eabfbdf6b546a383e8619acd9e0881', 'request_id': 'req-5da3766f-a1a5-4074-9258-d9cead6af13c'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:54.231529 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5da3766f-a1a5-4074-9258-d9cead6af13c admin admin] The token cache is being invalidate because project 76eabfbdf6b546a383e8619acd9e0881 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:33:54.231839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5da3766f-a1a5-4074-9258-d9cead6af13c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:54.240755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5da3766f-a1a5-4074-9258-d9cead6af13c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:54.242720 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 929/1853] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:33:53 2022] DELETE /identity/v3/projects/76eabfbdf6b546a383e8619acd9e0881 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:54.248984 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3766f-a1a5-4074-9258-d9cead6af13c admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:54.250297 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5da3766f-a1a5-4074-9258-d9cead6af13c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:54.327369 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 925/1854] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:33:54.331457 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a86cc4-ae12-4c42-be9f-f27b25140442 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:54.338264 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-a6a86cc4-ae12-4c42-be9f-f27b25140442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:54.399650 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3766f-a1a5-4074-9258-d9cead6af13c admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:54.417348 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:54.495831 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a86cc4-ae12-4c42-be9f-f27b25140442 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:54.510380 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c7034-8f0f-473e-b7aa-34fdd310467e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:54.511880 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c7034-8f0f-473e-b7aa-34fdd310467e admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:54.511880 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c7034-8f0f-473e-b7aa-34fdd310467e admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:54.511880 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c7034-8f0f-473e-b7aa-34fdd310467e admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:54.513804 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c7034-8f0f-473e-b7aa-34fdd310467e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:54.520050 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c7034-8f0f-473e-b7aa-34fdd310467e admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:54.581364 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe99fc-1708-4f21-bfe1-585b943549e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:54.583139 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe99fc-1708-4f21-bfe1-585b943549e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:54.583658 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe99fc-1708-4f21-bfe1-585b943549e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:54.590263 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe99fc-1708-4f21-bfe1-585b943549e2 admin admin] PATH_INFO: `/v3/users/15ec6365d3634355b1261ed00bcbc279` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:54.592539 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe99fc-1708-4f21-bfe1-585b943549e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15ec6365d3634355b1261ed00bcbc279)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:54.595927 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe99fc-1708-4f21-bfe1-585b943549e2 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:54.618405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 926/1855] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:33:54 2022] POST /identity/v3/projects => generated 387 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:54.624520 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c7034-8f0f-473e-b7aa-34fdd310467e admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:54.772961 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6abe99fc-1708-4f21-bfe1-585b943549e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15ec6365d3634355b1261ed00bcbc279'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:54.833885 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6abe99fc-1708-4f21-bfe1-585b943549e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15ec6365d3634355b1261ed00bcbc279'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:54.863454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6abe99fc-1708-4f21-bfe1-585b943549e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15ec6365d3634355b1261ed00bcbc279'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:54.864441 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6abe99fc-1708-4f21-bfe1-585b943549e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15ec6365d3634355b1261ed00bcbc279'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:54.865403 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6abe99fc-1708-4f21-bfe1-585b943549e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15ec6365d3634355b1261ed00bcbc279'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:54.888016 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c7034-8f0f-473e-b7aa-34fdd310467e admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:54.905497 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb83ae5-6607-4d78-b9aa-20e31238e2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:54.906750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb83ae5-6607-4d78-b9aa-20e31238e2d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:54.910618 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb83ae5-6607-4d78-b9aa-20e31238e2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:54.912476 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb83ae5-6607-4d78-b9aa-20e31238e2d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:54.915948 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb83ae5-6607-4d78-b9aa-20e31238e2d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:54.916493 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 930/1856] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:33:54 2022] DELETE /identity/v3/users/15ec6365d3634355b1261ed00bcbc279 => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:54.919406 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb83ae5-6607-4d78-b9aa-20e31238e2d9 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:54.927842 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:54.959743 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 927/1857] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:33:54 2022] POST /identity/v3/users => generated 300 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:54.959987 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe99fc-1708-4f21-bfe1-585b943549e2 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:54.959987 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-6abe99fc-1708-4f21-bfe1-585b943549e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:55.054281 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe99fc-1708-4f21-bfe1-585b943549e2 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:55.065003 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561e726-4f6e-428f-9155-b26b460f7e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:55.068469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561e726-4f6e-428f-9155-b26b460f7e31 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:55.068945 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561e726-4f6e-428f-9155-b26b460f7e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:55.069611 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561e726-4f6e-428f-9155-b26b460f7e31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:55.070806 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8561e726-4f6e-428f-9155-b26b460f7e31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:55.073516 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8561e726-4f6e-428f-9155-b26b460f7e31 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:55.094534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 931/1858] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:55.095914 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb83ae5-6607-4d78-b9aa-20e31238e2d9 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:55.107158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb83ae5-6607-4d78-b9aa-20e31238e2d9 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:55.113908 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f215fd-467d-4db5-bd9f-855179761c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:55.124108 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f215fd-467d-4db5-bd9f-855179761c8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:55.124693 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f215fd-467d-4db5-bd9f-855179761c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:55.124987 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f215fd-467d-4db5-bd9f-855179761c8e admin admin] PATH_INFO: `/v3/projects/e5d3739b11c84b0fa8c62872f1bb61e0/users/c094f88d4cf14d6ba9f9aae7a115b3c4/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:55.151636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f215fd-467d-4db5-bd9f-855179761c8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5d3739b11c84b0fa8c62872f1bb61e0, user_id=c094f88d4cf14d6ba9f9aae7a115b3c4, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:55.155395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f215fd-467d-4db5-bd9f-855179761c8e admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:55.162493 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561e726-4f6e-428f-9155-b26b460f7e31 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:55.168983 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-8561e726-4f6e-428f-9155-b26b460f7e31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:55.176069 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 928/1859] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:55 2022] PUT /identity/v3/projects/e5d3739b11c84b0fa8c62872f1bb61e0/users/c094f88d4cf14d6ba9f9aae7a115b3c4/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:55.180085 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f215fd-467d-4db5-bd9f-855179761c8e admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:55.194492 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561e726-4f6e-428f-9155-b26b460f7e31 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:55.250600 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:55.317893 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f215fd-467d-4db5-bd9f-855179761c8e admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:55.331002 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e7df3-41ad-476a-923f-7cd134bb2fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:55.331697 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e7df3-41ad-476a-923f-7cd134bb2fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:55.332138 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e7df3-41ad-476a-923f-7cd134bb2fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:55.332532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e7df3-41ad-476a-923f-7cd134bb2fd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:55.334973 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e7df3-41ad-476a-923f-7cd134bb2fd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:55.336840 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e7df3-41ad-476a-923f-7cd134bb2fd9 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:55.350805 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b3ed24-b9b3-4a3c-8847-d3cbf23e09b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:55.351815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 929/1860] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:55.355764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3ed24-b9b3-4a3c-8847-d3cbf23e09b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:55.356090 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3ed24-b9b3-4a3c-8847-d3cbf23e09b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:55.356419 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3ed24-b9b3-4a3c-8847-d3cbf23e09b7 admin admin] PATH_INFO: `/v3/projects/4116bb505734419888a2f3c40d29406b` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:55.356848 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e7df3-41ad-476a-923f-7cd134bb2fd9 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:55.358892 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b3ed24-b9b3-4a3c-8847-d3cbf23e09b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4116bb505734419888a2f3c40d29406b)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:55.360085 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b3ed24-b9b3-4a3c-8847-d3cbf23e09b7 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:55.373409 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e7df3-41ad-476a-923f-7cd134bb2fd9 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:55.379908 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45eab8d-3628-486f-9543-6376883fb2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:55.380975 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45eab8d-3628-486f-9543-6376883fb2e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:55.381325 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45eab8d-3628-486f-9543-6376883fb2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:55.382257 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45eab8d-3628-486f-9543-6376883fb2e1 admin admin] PATH_INFO: `/v3/projects/e5d3739b11c84b0fa8c62872f1bb61e0/users/c094f88d4cf14d6ba9f9aae7a115b3c4/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:55.387208 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45eab8d-3628-486f-9543-6376883fb2e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5d3739b11c84b0fa8c62872f1bb61e0, user_id=c094f88d4cf14d6ba9f9aae7a115b3c4, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:55.390479 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45eab8d-3628-486f-9543-6376883fb2e1 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:55.406015 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 930/1861] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:55 2022] PUT /identity/v3/projects/e5d3739b11c84b0fa8c62872f1bb61e0/users/c094f88d4cf14d6ba9f9aae7a115b3c4/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:33:55.418956 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da6506-ff8d-4534-b466-da3191ee5ce1 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:55.419259 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da6506-ff8d-4534-b466-da3191ee5ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:55.419672 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da6506-ff8d-4534-b466-da3191ee5ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:55.456228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-f9da6506-ff8d-4534-b466-da3191ee5ce1 None None] MFA Rules not processed for user `c094f88d4cf14d6ba9f9aae7a115b3c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:55.471256 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-f9da6506-ff8d-4534-b466-da3191ee5ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:55.474903 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a7b3ed24-b9b3-4a3c-8847-d3cbf23e09b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4116bb505734419888a2f3c40d29406b', 'request_id': 'req-a7b3ed24-b9b3-4a3c-8847-d3cbf23e09b7'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:55.491552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a7b3ed24-b9b3-4a3c-8847-d3cbf23e09b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4116bb505734419888a2f3c40d29406b', 'request_id': 'req-a7b3ed24-b9b3-4a3c-8847-d3cbf23e09b7'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:55.507176 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a7b3ed24-b9b3-4a3c-8847-d3cbf23e09b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4116bb505734419888a2f3c40d29406b', 'request_id': 'req-a7b3ed24-b9b3-4a3c-8847-d3cbf23e09b7'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:55.540713 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a7b3ed24-b9b3-4a3c-8847-d3cbf23e09b7 admin admin] The token cache is being invalidate because project 4116bb505734419888a2f3c40d29406b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:33:55.540945 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a7b3ed24-b9b3-4a3c-8847-d3cbf23e09b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:55.542359 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a7b3ed24-b9b3-4a3c-8847-d3cbf23e09b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:55.546046 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 932/1862] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:33:55 2022] DELETE /identity/v3/projects/4116bb505734419888a2f3c40d29406b => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:33:55.549951 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b3ed24-b9b3-4a3c-8847-d3cbf23e09b7 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:55.556427 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a7b3ed24-b9b3-4a3c-8847-d3cbf23e09b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:55.610053 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 931/1863] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:55 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:55.619194 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da6506-ff8d-4534-b466-da3191ee5ce1 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:55.621342 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-f9da6506-ff8d-4534-b466-da3191ee5ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:55.691331 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:55.698326 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b3ed24-b9b3-4a3c-8847-d3cbf23e09b7 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:55.715491 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec63a3-7aad-485e-b1f6-a2bc3937eedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:55.716530 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec63a3-7aad-485e-b1f6-a2bc3937eedc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:55.716815 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec63a3-7aad-485e-b1f6-a2bc3937eedc admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:55.717096 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec63a3-7aad-485e-b1f6-a2bc3937eedc admin admin] PATH_INFO: `/v3/users/ea065ecf98b941beb1b7a2ec68eb17da` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:55.720730 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bec63a3-7aad-485e-b1f6-a2bc3937eedc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea065ecf98b941beb1b7a2ec68eb17da)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:55.723299 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bec63a3-7aad-485e-b1f6-a2bc3937eedc admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:55.733080 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da6506-ff8d-4534-b466-da3191ee5ce1 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:55.740509 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21291ad-48b1-4074-93bd-ea6a55b9141f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:55.741342 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21291ad-48b1-4074-93bd-ea6a55b9141f service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:55.741823 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21291ad-48b1-4074-93bd-ea6a55b9141f service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:55.742357 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21291ad-48b1-4074-93bd-ea6a55b9141f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:55.757390 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21291ad-48b1-4074-93bd-ea6a55b9141f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:55.760039 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21291ad-48b1-4074-93bd-ea6a55b9141f service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:55.772536 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 932/1864] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:55 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:55.826799 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21291ad-48b1-4074-93bd-ea6a55b9141f service cinder] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:55.828405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-f21291ad-48b1-4074-93bd-ea6a55b9141f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:55.873883 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21291ad-48b1-4074-93bd-ea6a55b9141f service cinder] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:55.879595 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec85fdf-739f-4966-9acb-e87bc1daa4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:55.880693 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec85fdf-739f-4966-9acb-e87bc1daa4ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:55.880986 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec85fdf-739f-4966-9acb-e87bc1daa4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:55.882103 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec85fdf-739f-4966-9acb-e87bc1daa4ef admin admin] PATH_INFO: `/v3/users/690fcaa67b5e469a8f32a748f837fada` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:55.884384 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec85fdf-739f-4966-9acb-e87bc1daa4ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=690fcaa67b5e469a8f32a748f837fada)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:55.884923 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3bec63a3-7aad-485e-b1f6-a2bc3937eedc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea065ecf98b941beb1b7a2ec68eb17da'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:55.886594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec85fdf-739f-4966-9acb-e87bc1daa4ef admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:55.906931 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:55.927834 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3bec63a3-7aad-485e-b1f6-a2bc3937eedc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea065ecf98b941beb1b7a2ec68eb17da'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:55.946997 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3bec63a3-7aad-485e-b1f6-a2bc3937eedc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea065ecf98b941beb1b7a2ec68eb17da'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:55.948019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3bec63a3-7aad-485e-b1f6-a2bc3937eedc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea065ecf98b941beb1b7a2ec68eb17da'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:55.949005 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3bec63a3-7aad-485e-b1f6-a2bc3937eedc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea065ecf98b941beb1b7a2ec68eb17da'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:55.979956 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2ec85fdf-739f-4966-9acb-e87bc1daa4ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '690fcaa67b5e469a8f32a748f837fada'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:55.980670 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2ec85fdf-739f-4966-9acb-e87bc1daa4ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '690fcaa67b5e469a8f32a748f837fada'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:55.981522 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 933/1865] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:33:55 2022] DELETE /identity/v3/users/ea065ecf98b941beb1b7a2ec68eb17da => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:55.995630 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2ec85fdf-739f-4966-9acb-e87bc1daa4ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '690fcaa67b5e469a8f32a748f837fada'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:56.058106 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2ec85fdf-739f-4966-9acb-e87bc1daa4ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '690fcaa67b5e469a8f32a748f837fada'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:56.073581 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2ec85fdf-739f-4966-9acb-e87bc1daa4ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '690fcaa67b5e469a8f32a748f837fada'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:56.083728 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec63a3-7aad-485e-b1f6-a2bc3937eedc admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:56.085581 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3bec63a3-7aad-485e-b1f6-a2bc3937eedc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:56.090497 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 933/1866] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:33:55 2022] DELETE /identity/v3/users/690fcaa67b5e469a8f32a748f837fada => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:56.093867 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2faeb91-4664-488e-aa92-5840b5148588 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:56.094167 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2faeb91-4664-488e-aa92-5840b5148588 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:56.094508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2faeb91-4664-488e-aa92-5840b5148588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:56.147192 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-c2faeb91-4664-488e-aa92-5840b5148588 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:56.156227 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-c2faeb91-4664-488e-aa92-5840b5148588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:56.269954 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec63a3-7aad-485e-b1f6-a2bc3937eedc admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:56.341006 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 934/1867] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:33:56.374327 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f660f46-0036-4d21-9532-e6fcb9505c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:56.376003 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f660f46-0036-4d21-9532-e6fcb9505c89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:56.376424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f660f46-0036-4d21-9532-e6fcb9505c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:56.377451 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f660f46-0036-4d21-9532-e6fcb9505c89 admin admin] PATH_INFO: `/v3/projects/3fe2b414d769416b834baf5c97ef6fc5` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:56.378114 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f660f46-0036-4d21-9532-e6fcb9505c89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fe2b414d769416b834baf5c97ef6fc5)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:56.384802 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f660f46-0036-4d21-9532-e6fcb9505c89 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:56.443495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2faeb91-4664-488e-aa92-5840b5148588 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:56.445022 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-c2faeb91-4664-488e-aa92-5840b5148588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:56.472726 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-0f660f46-0036-4d21-9532-e6fcb9505c89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fe2b414d769416b834baf5c97ef6fc5', 'request_id': 'req-0f660f46-0036-4d21-9532-e6fcb9505c89'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:56.518749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-0f660f46-0036-4d21-9532-e6fcb9505c89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fe2b414d769416b834baf5c97ef6fc5', 'request_id': 'req-0f660f46-0036-4d21-9532-e6fcb9505c89'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:56.544915 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-0f660f46-0036-4d21-9532-e6fcb9505c89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fe2b414d769416b834baf5c97ef6fc5', 'request_id': 'req-0f660f46-0036-4d21-9532-e6fcb9505c89'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:56.605946 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-0f660f46-0036-4d21-9532-e6fcb9505c89 admin admin] The token cache is being invalidate because project 3fe2b414d769416b834baf5c97ef6fc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:33:56.606927 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-0f660f46-0036-4d21-9532-e6fcb9505c89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:56.608015 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-0f660f46-0036-4d21-9532-e6fcb9505c89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:56.611348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2faeb91-4664-488e-aa92-5840b5148588 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:56.616917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:56.618570 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 934/1868] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:33:56 2022] DELETE /identity/v3/projects/3fe2b414d769416b834baf5c97ef6fc5 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:56.665806 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113c830-7145-4123-88a0-dbde46fe8a8b None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:56.666634 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113c830-7145-4123-88a0-dbde46fe8a8b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:56.666634 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a113c830-7145-4123-88a0-dbde46fe8a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:56.718353 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a113c830-7145-4123-88a0-dbde46fe8a8b None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:56.721081 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a113c830-7145-4123-88a0-dbde46fe8a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:56.729308 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:56.737078 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7cb91e-bf0e-44b9-b8c5-c219e654c8f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:56.737762 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7cb91e-bf0e-44b9-b8c5-c219e654c8f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:56.738239 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7cb91e-bf0e-44b9-b8c5-c219e654c8f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:56.738517 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7cb91e-bf0e-44b9-b8c5-c219e654c8f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:56.740780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-1b7cb91e-bf0e-44b9-b8c5-c219e654c8f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:56.813658 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7cb91e-bf0e-44b9-b8c5-c219e654c8f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:56.815911 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7cb91e-bf0e-44b9-b8c5-c219e654c8f8 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:56.830226 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 935/1869] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:56.831108 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7cb91e-bf0e-44b9-b8c5-c219e654c8f8 service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:56.833257 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-1b7cb91e-bf0e-44b9-b8c5-c219e654c8f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:56.847673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7cb91e-bf0e-44b9-b8c5-c219e654c8f8 service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:56.860585 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6c00f-75d8-4f1f-acfe-dcc24e43dbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:56.861487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6c00f-75d8-4f1f-acfe-dcc24e43dbd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:56.861780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6c00f-75d8-4f1f-acfe-dcc24e43dbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:56.862306 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6c00f-75d8-4f1f-acfe-dcc24e43dbd3 admin admin] PATH_INFO: `/v3/projects/18333e2a50f94a6ab0e819609de48543` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:56.864021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe6c00f-75d8-4f1f-acfe-dcc24e43dbd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18333e2a50f94a6ab0e819609de48543)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:56.866762 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe6c00f-75d8-4f1f-acfe-dcc24e43dbd3 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:56.889815 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 935/1870] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:56.891602 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6555c3-f9de-4958-96c9-6a264400334d None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:56.891894 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6555c3-f9de-4958-96c9-6a264400334d None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:56.892183 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6555c3-f9de-4958-96c9-6a264400334d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:56.918165 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-0e6555c3-f9de-4958-96c9-6a264400334d None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:56.920397 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-0e6555c3-f9de-4958-96c9-6a264400334d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:56.930122 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-abe6c00f-75d8-4f1f-acfe-dcc24e43dbd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18333e2a50f94a6ab0e819609de48543', 'request_id': 'req-abe6c00f-75d8-4f1f-acfe-dcc24e43dbd3'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:56.948064 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-abe6c00f-75d8-4f1f-acfe-dcc24e43dbd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18333e2a50f94a6ab0e819609de48543', 'request_id': 'req-abe6c00f-75d8-4f1f-acfe-dcc24e43dbd3'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:56.956974 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-abe6c00f-75d8-4f1f-acfe-dcc24e43dbd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18333e2a50f94a6ab0e819609de48543', 'request_id': 'req-abe6c00f-75d8-4f1f-acfe-dcc24e43dbd3'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:56.970665 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-abe6c00f-75d8-4f1f-acfe-dcc24e43dbd3 admin admin] The token cache is being invalidate because project 18333e2a50f94a6ab0e819609de48543 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:33:56.971820 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-abe6c00f-75d8-4f1f-acfe-dcc24e43dbd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:56.972865 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-abe6c00f-75d8-4f1f-acfe-dcc24e43dbd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:56.975430 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 936/1871] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:33:56 2022] DELETE /identity/v3/projects/18333e2a50f94a6ab0e819609de48543 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:56.983339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd08e2-248d-469b-8c5f-8bf10b35d417 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:56.983727 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd08e2-248d-469b-8c5f-8bf10b35d417 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:56.984109 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd08e2-248d-469b-8c5f-8bf10b35d417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:57.007383 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 936/1872] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:57.008456 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6555c3-f9de-4958-96c9-6a264400334d None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:57.018381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-17bd08e2-248d-469b-8c5f-8bf10b35d417 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:57.019793 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-17bd08e2-248d-469b-8c5f-8bf10b35d417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:57.030881 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6555c3-f9de-4958-96c9-6a264400334d None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:57.037009 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 937/1873] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:57.045276 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60581e8f-52bd-4e58-bc4c-9bad3ab9db6b None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:57.045893 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66113d9-903d-46e2-83eb-7c626eda4e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:57.046058 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60581e8f-52bd-4e58-bc4c-9bad3ab9db6b None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:57.046416 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60581e8f-52bd-4e58-bc4c-9bad3ab9db6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:57.046578 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66113d9-903d-46e2-83eb-7c626eda4e95 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:57.046879 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66113d9-903d-46e2-83eb-7c626eda4e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:57.047144 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66113d9-903d-46e2-83eb-7c626eda4e95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:57.047885 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66113d9-903d-46e2-83eb-7c626eda4e95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:33:57.048329 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66113d9-903d-46e2-83eb-7c626eda4e95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:57.049963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66113d9-903d-46e2-83eb-7c626eda4e95 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:57.066622 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 937/1874] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:33:57.073350 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66113d9-903d-46e2-83eb-7c626eda4e95 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:57.080583 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-60581e8f-52bd-4e58-bc4c-9bad3ab9db6b None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:57.081872 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-60581e8f-52bd-4e58-bc4c-9bad3ab9db6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:57.093688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66113d9-903d-46e2-83eb-7c626eda4e95 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:57.104140 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c8c0b-2cf4-44c6-b2bb-667c05793480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:57.104876 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c8c0b-2cf4-44c6-b2bb-667c05793480 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:57.105163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c8c0b-2cf4-44c6-b2bb-667c05793480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:57.105442 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c8c0b-2cf4-44c6-b2bb-667c05793480 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:57.107288 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c8c0b-2cf4-44c6-b2bb-667c05793480 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:57.109349 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 938/1875] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:33:57.114083 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60581e8f-52bd-4e58-bc4c-9bad3ab9db6b None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:57.119885 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c8c0b-2cf4-44c6-b2bb-667c05793480 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:57.132685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60581e8f-52bd-4e58-bc4c-9bad3ab9db6b None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:57.143565 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a233c4-0237-4753-90fc-2c272ed43f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:57.144882 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a233c4-0237-4753-90fc-2c272ed43f55 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:57.145343 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a233c4-0237-4753-90fc-2c272ed43f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:57.145627 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a233c4-0237-4753-90fc-2c272ed43f55 admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:57.146586 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a233c4-0237-4753-90fc-2c272ed43f55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:33:57.146888 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a233c4-0237-4753-90fc-2c272ed43f55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:57.148605 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a233c4-0237-4753-90fc-2c272ed43f55 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:57.168336 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 939/1877] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:33:57.168336 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a233c4-0237-4753-90fc-2c272ed43f55 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:57.169598 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 938/1877] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:33:57 2022] POST /identity/v3/projects => generated 367 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:57.170600 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c8c0b-2cf4-44c6-b2bb-667c05793480 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:57.193099 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a233c4-0237-4753-90fc-2c272ed43f55 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:57.210877 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:57.314733 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c8c0b-2cf4-44c6-b2bb-667c05793480 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:57.337758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d03e0ae-afe8-4eee-ad70-e5a01cb82911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:57.339826 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03e0ae-afe8-4eee-ad70-e5a01cb82911 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:57.340122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03e0ae-afe8-4eee-ad70-e5a01cb82911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:57.340432 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03e0ae-afe8-4eee-ad70-e5a01cb82911 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:57.341156 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d03e0ae-afe8-4eee-ad70-e5a01cb82911 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:57.343802 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d03e0ae-afe8-4eee-ad70-e5a01cb82911 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:57.358231 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a009af-6c37-4b1a-b01c-30b71c99b8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:57.358231 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a009af-6c37-4b1a-b01c-30b71c99b8df admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:57.358899 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a009af-6c37-4b1a-b01c-30b71c99b8df admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:57.359084 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a009af-6c37-4b1a-b01c-30b71c99b8df admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:57.359949 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a009af-6c37-4b1a-b01c-30b71c99b8df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:57.361867 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a009af-6c37-4b1a-b01c-30b71c99b8df admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:57.407942 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 939/1878] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:33:57 2022] POST /identity/v3/projects => generated 371 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:57.417323 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d03e0ae-afe8-4eee-ad70-e5a01cb82911 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:57.422732 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:57.426149 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 940/1879] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:33:57 2022] POST /identity/v3/users => generated 284 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:57.431519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a009af-6c37-4b1a-b01c-30b71c99b8df admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:57.459620 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a009af-6c37-4b1a-b01c-30b71c99b8df admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:57.527500 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d03e0ae-afe8-4eee-ad70-e5a01cb82911 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:57.541439 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbebd62-f006-4d27-a42b-9460707212eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:57.544141 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbebd62-f006-4d27-a42b-9460707212eb admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:57.544644 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbebd62-f006-4d27-a42b-9460707212eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:57.545477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbebd62-f006-4d27-a42b-9460707212eb admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:57.546523 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbebd62-f006-4d27-a42b-9460707212eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:57.549010 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbebd62-f006-4d27-a42b-9460707212eb admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:57.597579 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 940/1880] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:33:57 2022] POST /identity/v3/users => generated 286 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:57.599955 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbebd62-f006-4d27-a42b-9460707212eb admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:57.622362 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbebd62-f006-4d27-a42b-9460707212eb admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:57.632465 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65489b-affa-4ec3-8587-34bc19973ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:57.638375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65489b-affa-4ec3-8587-34bc19973ae0 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:57.638933 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65489b-affa-4ec3-8587-34bc19973ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:57.639399 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05a597-a62a-4e14-98d9-3bfb8416eff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:57.640733 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb65489b-affa-4ec3-8587-34bc19973ae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:57.641184 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05a597-a62a-4e14-98d9-3bfb8416eff0 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:57.641938 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05a597-a62a-4e14-98d9-3bfb8416eff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:57.642218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65489b-affa-4ec3-8587-34bc19973ae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:57.643009 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05a597-a62a-4e14-98d9-3bfb8416eff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:57.645416 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb65489b-affa-4ec3-8587-34bc19973ae0 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:57.646099 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce05a597-a62a-4e14-98d9-3bfb8416eff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:57.651409 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce05a597-a62a-4e14-98d9-3bfb8416eff0 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:57.678644 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 941/1881] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:57.683812 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65489b-affa-4ec3-8587-34bc19973ae0 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:57.685377 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 941/1882] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:57.689586 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05a597-a62a-4e14-98d9-3bfb8416eff0 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:57.702752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05a597-a62a-4e14-98d9-3bfb8416eff0 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:57.707864 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb65489b-affa-4ec3-8587-34bc19973ae0 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:57.708396 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912d9ab-7c99-4a85-8e42-66440fda198c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:57.709082 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912d9ab-7c99-4a85-8e42-66440fda198c admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:57.709369 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912d9ab-7c99-4a85-8e42-66440fda198c admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:57.709652 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912d9ab-7c99-4a85-8e42-66440fda198c admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:57.710434 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0912d9ab-7c99-4a85-8e42-66440fda198c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:57.712154 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0912d9ab-7c99-4a85-8e42-66440fda198c admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:57.725119 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32dcab-a707-4c63-bb77-8777419c84b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:57.726231 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32dcab-a707-4c63-bb77-8777419c84b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:57.726731 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32dcab-a707-4c63-bb77-8777419c84b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:57.727744 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32dcab-a707-4c63-bb77-8777419c84b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:57.730991 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c32dcab-a707-4c63-bb77-8777419c84b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:57.731256 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 942/1883] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:57.732273 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912d9ab-7c99-4a85-8e42-66440fda198c admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:57.735158 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c32dcab-a707-4c63-bb77-8777419c84b9 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:57.743301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912d9ab-7c99-4a85-8e42-66440fda198c admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:57.751389 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97a9ab-1b7a-4e37-9c50-3918a67c82d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:57.752352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97a9ab-1b7a-4e37-9c50-3918a67c82d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:57.752637 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97a9ab-1b7a-4e37-9c50-3918a67c82d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:57.752961 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97a9ab-1b7a-4e37-9c50-3918a67c82d8 admin admin] PATH_INFO: `/v3/projects/7cb4a58a707f4ddca6930fd35a34c30c/users/600c873e0d1645268e49ec8144fc913c/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:57.763746 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 942/1884] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:33:57.772843 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c97a9ab-1b7a-4e37-9c50-3918a67c82d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cb4a58a707f4ddca6930fd35a34c30c, user_id=600c873e0d1645268e49ec8144fc913c, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:57.774261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32dcab-a707-4c63-bb77-8777419c84b9 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:57.777604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c97a9ab-1b7a-4e37-9c50-3918a67c82d8 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:57.799661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32dcab-a707-4c63-bb77-8777419c84b9 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:57.815665 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 943/1885] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:57 2022] PUT /identity/v3/projects/7cb4a58a707f4ddca6930fd35a34c30c/users/600c873e0d1645268e49ec8144fc913c/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:57.818734 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0af61-ceaa-4558-b562-ac96377f89bb None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:57.818734 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0af61-ceaa-4558-b562-ac96377f89bb None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:57.818978 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0af61-ceaa-4558-b562-ac96377f89bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:57.855904 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-dfa0af61-ceaa-4558-b562-ac96377f89bb None None] MFA Rules not processed for user `600c873e0d1645268e49ec8144fc913c`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:57.857512 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-dfa0af61-ceaa-4558-b562-ac96377f89bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:57.946756 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:57.951842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e66c7-0257-435f-a18d-9348670a5a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:57.951842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e66c7-0257-435f-a18d-9348670a5a34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:57.951842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e66c7-0257-435f-a18d-9348670a5a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:57.951842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e66c7-0257-435f-a18d-9348670a5a34 admin admin] PATH_INFO: `/v3/projects/5add1b56848d44eda7b0dae0b34bb67e/users/6490654f4f4543e6a1f51cfaf543176c/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:57.978302 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 944/1886] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:57 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:57.979330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0af61-ceaa-4558-b562-ac96377f89bb None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:57.980834 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-dfa0af61-ceaa-4558-b562-ac96377f89bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:57.996503 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0af61-ceaa-4558-b562-ac96377f89bb None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:57.998763 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e66c7-0257-435f-a18d-9348670a5a34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5add1b56848d44eda7b0dae0b34bb67e, user_id=6490654f4f4543e6a1f51cfaf543176c, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:58.001496 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e66c7-0257-435f-a18d-9348670a5a34 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:58.001659 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50142824-6f44-4a2e-81ef-8167c1cc5469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:58.002435 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50142824-6f44-4a2e-81ef-8167c1cc5469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:58.002719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50142824-6f44-4a2e-81ef-8167c1cc5469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:58.003001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50142824-6f44-4a2e-81ef-8167c1cc5469 admin admin] PATH_INFO: `/v3/projects/9aa39311ac3c442bbe55c6f32d0b0680` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:58.005128 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50142824-6f44-4a2e-81ef-8167c1cc5469 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9aa39311ac3c442bbe55c6f32d0b0680)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:58.008001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50142824-6f44-4a2e-81ef-8167c1cc5469 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:58.024968 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 943/1887] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:57 2022] PUT /identity/v3/projects/5add1b56848d44eda7b0dae0b34bb67e/users/6490654f4f4543e6a1f51cfaf543176c/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:58.025500 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e66c7-0257-435f-a18d-9348670a5a34 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:58.083700 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-50142824-6f44-4a2e-81ef-8167c1cc5469 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9aa39311ac3c442bbe55c6f32d0b0680', 'request_id': 'req-50142824-6f44-4a2e-81ef-8167c1cc5469'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:58.099673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-50142824-6f44-4a2e-81ef-8167c1cc5469 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9aa39311ac3c442bbe55c6f32d0b0680', 'request_id': 'req-50142824-6f44-4a2e-81ef-8167c1cc5469'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:58.132075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-50142824-6f44-4a2e-81ef-8167c1cc5469 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9aa39311ac3c442bbe55c6f32d0b0680', 'request_id': 'req-50142824-6f44-4a2e-81ef-8167c1cc5469'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:58.156680 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:58.161042 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-50142824-6f44-4a2e-81ef-8167c1cc5469 admin admin] The token cache is being invalidate because project 9aa39311ac3c442bbe55c6f32d0b0680 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:33:58.161752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-50142824-6f44-4a2e-81ef-8167c1cc5469 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:58.162697 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-50142824-6f44-4a2e-81ef-8167c1cc5469 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:58.164887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 945/1888] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:33:57 2022] DELETE /identity/v3/projects/9aa39311ac3c442bbe55c6f32d0b0680 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:58.166459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019af444-71cb-46ba-88e5-8e98f82b6eb1 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:58.166818 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019af444-71cb-46ba-88e5-8e98f82b6eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:58.167702 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019af444-71cb-46ba-88e5-8e98f82b6eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:58.189319 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e66c7-0257-435f-a18d-9348670a5a34 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:58.213145 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-019af444-71cb-46ba-88e5-8e98f82b6eb1 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:58.214396 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-019af444-71cb-46ba-88e5-8e98f82b6eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:58.275940 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac920dff-35e2-4ea7-9372-3cd7ba33ae68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:58.278944 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac920dff-35e2-4ea7-9372-3cd7ba33ae68 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:58.279575 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac920dff-35e2-4ea7-9372-3cd7ba33ae68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:58.282959 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac920dff-35e2-4ea7-9372-3cd7ba33ae68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:58.283710 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac920dff-35e2-4ea7-9372-3cd7ba33ae68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:58.288355 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac920dff-35e2-4ea7-9372-3cd7ba33ae68 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:58.339579 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 944/1889] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:33:58 2022] POST /identity/v3/projects => generated 367 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:58.341254 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac920dff-35e2-4ea7-9372-3cd7ba33ae68 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:58.344745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ac920dff-35e2-4ea7-9372-3cd7ba33ae68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:58.351558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 946/1890] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:58.353305 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019af444-71cb-46ba-88e5-8e98f82b6eb1 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:58.360421 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-019af444-71cb-46ba-88e5-8e98f82b6eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:58.398824 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019af444-71cb-46ba-88e5-8e98f82b6eb1 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:58.489119 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac920dff-35e2-4ea7-9372-3cd7ba33ae68 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:58.501586 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45da4ae-1e72-4622-82fb-3608bf766fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:58.503448 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45da4ae-1e72-4622-82fb-3608bf766fbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:58.504661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45da4ae-1e72-4622-82fb-3608bf766fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:58.505147 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45da4ae-1e72-4622-82fb-3608bf766fbb admin admin] PATH_INFO: `/v3/users/c094f88d4cf14d6ba9f9aae7a115b3c4` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:58.507326 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45da4ae-1e72-4622-82fb-3608bf766fbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c094f88d4cf14d6ba9f9aae7a115b3c4)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:58.511955 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45da4ae-1e72-4622-82fb-3608bf766fbb admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:58.523334 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b426f6-ce35-44e0-95a7-c5dadba0b9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:58.524126 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b426f6-ce35-44e0-95a7-c5dadba0b9a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:58.525101 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b426f6-ce35-44e0-95a7-c5dadba0b9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:58.526021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b426f6-ce35-44e0-95a7-c5dadba0b9a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:58.526901 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b426f6-ce35-44e0-95a7-c5dadba0b9a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:58.529397 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b426f6-ce35-44e0-95a7-c5dadba0b9a3 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:58.568844 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 947/1891] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:33:58 2022] POST /identity/v3/users => generated 290 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:33:58.570089 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b426f6-ce35-44e0-95a7-c5dadba0b9a3 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:58.573349 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-18b426f6-ce35-44e0-95a7-c5dadba0b9a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:58.588715 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:58.639905 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-b45da4ae-1e72-4622-82fb-3608bf766fbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c094f88d4cf14d6ba9f9aae7a115b3c4'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:58.675877 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-b45da4ae-1e72-4622-82fb-3608bf766fbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c094f88d4cf14d6ba9f9aae7a115b3c4'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:58.689457 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-b45da4ae-1e72-4622-82fb-3608bf766fbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c094f88d4cf14d6ba9f9aae7a115b3c4'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:58.690257 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-b45da4ae-1e72-4622-82fb-3608bf766fbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c094f88d4cf14d6ba9f9aae7a115b3c4'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:58.690930 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-b45da4ae-1e72-4622-82fb-3608bf766fbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c094f88d4cf14d6ba9f9aae7a115b3c4'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:58.705516 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b426f6-ce35-44e0-95a7-c5dadba0b9a3 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:58.707511 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 945/1892] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:33:58 2022] DELETE /identity/v3/users/c094f88d4cf14d6ba9f9aae7a115b3c4 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:58.708734 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45da4ae-1e72-4622-82fb-3608bf766fbb admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:58.723382 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b45da4ae-1e72-4622-82fb-3608bf766fbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:58.749029 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:58.789641 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec0884-4136-4614-826e-6f8cf4dd25ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:58.791056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec0884-4136-4614-826e-6f8cf4dd25ec service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:58.791344 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec0884-4136-4614-826e-6f8cf4dd25ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:58.791629 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec0884-4136-4614-826e-6f8cf4dd25ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:58.796253 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-d8ec0884-4136-4614-826e-6f8cf4dd25ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:58.872484 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45da4ae-1e72-4622-82fb-3608bf766fbb admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:58.880030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc40c23-ee04-40af-8c2a-b15aab9abd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:58.880857 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc40c23-ee04-40af-8c2a-b15aab9abd88 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:58.881218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc40c23-ee04-40af-8c2a-b15aab9abd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:58.881564 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc40c23-ee04-40af-8c2a-b15aab9abd88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:58.882364 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc40c23-ee04-40af-8c2a-b15aab9abd88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:58.885609 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc40c23-ee04-40af-8c2a-b15aab9abd88 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:58.907028 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 946/1893] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:58.911892 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc40c23-ee04-40af-8c2a-b15aab9abd88 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:58.927042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc40c23-ee04-40af-8c2a-b15aab9abd88 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:58.933032 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec0884-4136-4614-826e-6f8cf4dd25ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:58.934651 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec0884-4136-4614-826e-6f8cf4dd25ec service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:58.936429 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452fe3d7-afd3-455a-9fb3-7a2e6bda23bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:58.937351 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452fe3d7-afd3-455a-9fb3-7a2e6bda23bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:58.939110 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452fe3d7-afd3-455a-9fb3-7a2e6bda23bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:58.939398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452fe3d7-afd3-455a-9fb3-7a2e6bda23bf admin admin] PATH_INFO: `/v3/projects/5ecc73eaacc4455e93cfba924de6e3a7/users/d8e336fdfc1146b6b960ec7fb50f0ce3/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:58.960106 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 948/1894] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:33:58.963788 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452fe3d7-afd3-455a-9fb3-7a2e6bda23bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ecc73eaacc4455e93cfba924de6e3a7, user_id=d8e336fdfc1146b6b960ec7fb50f0ce3, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:58.967160 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452fe3d7-afd3-455a-9fb3-7a2e6bda23bf admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:58.987389 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 947/1895] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:58 2022] PUT /identity/v3/projects/5ecc73eaacc4455e93cfba924de6e3a7/users/d8e336fdfc1146b6b960ec7fb50f0ce3/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:58.991301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec0884-4136-4614-826e-6f8cf4dd25ec service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:59.004903 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec0884-4136-4614-826e-6f8cf4dd25ec service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:59.100636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730aa1a9-1cbe-4aba-9c02-ff83fd5339ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:59.101422 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730aa1a9-1cbe-4aba-9c02-ff83fd5339ac admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:59.101721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730aa1a9-1cbe-4aba-9c02-ff83fd5339ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:59.102006 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730aa1a9-1cbe-4aba-9c02-ff83fd5339ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:59.102788 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730aa1a9-1cbe-4aba-9c02-ff83fd5339ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:59.104493 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730aa1a9-1cbe-4aba-9c02-ff83fd5339ac admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:59.118838 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 949/1896] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:33:59.123209 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452fe3d7-afd3-455a-9fb3-7a2e6bda23bf admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:59.137343 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452fe3d7-afd3-455a-9fb3-7a2e6bda23bf admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:59.159588 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4bf8ea-c32c-4bfa-aa07-b5b560d507f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:59.160555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bf8ea-c32c-4bfa-aa07-b5b560d507f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:59.160972 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bf8ea-c32c-4bfa-aa07-b5b560d507f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:59.161128 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bf8ea-c32c-4bfa-aa07-b5b560d507f3 admin admin] PATH_INFO: `/v3/projects/5ecc73eaacc4455e93cfba924de6e3a7/users/d8e336fdfc1146b6b960ec7fb50f0ce3/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:59.165538 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4bf8ea-c32c-4bfa-aa07-b5b560d507f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ecc73eaacc4455e93cfba924de6e3a7, user_id=d8e336fdfc1146b6b960ec7fb50f0ce3, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:59.178091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4bf8ea-c32c-4bfa-aa07-b5b560d507f3 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:59.215770 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 948/1897] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:33:59 2022] PUT /identity/v3/projects/5ecc73eaacc4455e93cfba924de6e3a7/users/d8e336fdfc1146b6b960ec7fb50f0ce3/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:33:59.219498 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba65dd4-3bc5-4cb3-b781-e21e61bf38e8 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:59.219806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba65dd4-3bc5-4cb3-b781-e21e61bf38e8 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:59.220106 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba65dd4-3bc5-4cb3-b781-e21e61bf38e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:59.240609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:59.268673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-fba65dd4-3bc5-4cb3-b781-e21e61bf38e8 None None] MFA Rules not processed for user `d8e336fdfc1146b6b960ec7fb50f0ce3`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:59.270342 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-fba65dd4-3bc5-4cb3-b781-e21e61bf38e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:59.370256 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4bf8ea-c32c-4bfa-aa07-b5b560d507f3 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:59.373511 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-0d4bf8ea-c32c-4bfa-aa07-b5b560d507f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:59.490469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 950/1898] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:59 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:59.515424 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba65dd4-3bc5-4cb3-b781-e21e61bf38e8 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:33:59.525553 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-fba65dd4-3bc5-4cb3-b781-e21e61bf38e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:59.537105 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4bf8ea-c32c-4bfa-aa07-b5b560d507f3 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:59.544886 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432eb426-6162-48e6-a54e-ca1ff061bff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:59.546794 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432eb426-6162-48e6-a54e-ca1ff061bff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:59.547357 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432eb426-6162-48e6-a54e-ca1ff061bff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:59.547861 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432eb426-6162-48e6-a54e-ca1ff061bff9 admin admin] PATH_INFO: `/v3/projects/e5d3739b11c84b0fa8c62872f1bb61e0` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:59.549512 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432eb426-6162-48e6-a54e-ca1ff061bff9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5d3739b11c84b0fa8c62872f1bb61e0)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:59.552640 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432eb426-6162-48e6-a54e-ca1ff061bff9 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:59.642627 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba65dd4-3bc5-4cb3-b781-e21e61bf38e8 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:33:59.658953 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-432eb426-6162-48e6-a54e-ca1ff061bff9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5d3739b11c84b0fa8c62872f1bb61e0', 'request_id': 'req-432eb426-6162-48e6-a54e-ca1ff061bff9'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:59.680853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-432eb426-6162-48e6-a54e-ca1ff061bff9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5d3739b11c84b0fa8c62872f1bb61e0', 'request_id': 'req-432eb426-6162-48e6-a54e-ca1ff061bff9'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:59.694701 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-432eb426-6162-48e6-a54e-ca1ff061bff9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5d3739b11c84b0fa8c62872f1bb61e0', 'request_id': 'req-432eb426-6162-48e6-a54e-ca1ff061bff9'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:59.698677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:59.716912 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-432eb426-6162-48e6-a54e-ca1ff061bff9 admin admin] The token cache is being invalidate because project e5d3739b11c84b0fa8c62872f1bb61e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:33:59.717307 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-432eb426-6162-48e6-a54e-ca1ff061bff9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:59.719088 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-432eb426-6162-48e6-a54e-ca1ff061bff9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:33:59.721433 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 949/1899] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:33:59 2022] DELETE /identity/v3/projects/e5d3739b11c84b0fa8c62872f1bb61e0 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:33:59.726263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1296f3-c8d4-41a3-a7da-065f9166bd8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:33:59.726992 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1296f3-c8d4-41a3-a7da-065f9166bd8c service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:59.727856 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1296f3-c8d4-41a3-a7da-065f9166bd8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:59.728144 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1296f3-c8d4-41a3-a7da-065f9166bd8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:59.728816 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8e500-e6b6-4593-ba52-c07849101ecd None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:59.728999 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8e500-e6b6-4593-ba52-c07849101ecd None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:59.729293 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8e500-e6b6-4593-ba52-c07849101ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:59.730530 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-4d1296f3-c8d4-41a3-a7da-065f9166bd8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:59.754232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-3cc8e500-e6b6-4593-ba52-c07849101ecd None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:59.755872 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3cc8e500-e6b6-4593-ba52-c07849101ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:59.876063 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 950/1900] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:59.890050 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1296f3-c8d4-41a3-a7da-065f9166bd8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:33:59.891035 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e8f64-f648-4e96-83fa-9cc88e7f728e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:33:59.892032 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1296f3-c8d4-41a3-a7da-065f9166bd8c service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:33:59.898214 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:33:59.900692 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e8f64-f648-4e96-83fa-9cc88e7f728e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:33:59.901328 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e8f64-f648-4e96-83fa-9cc88e7f728e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:33:59.918528 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 951/1901] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:33:59 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:33:59.959475 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-583e8f64-f648-4e96-83fa-9cc88e7f728e None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:33:59.962677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-583e8f64-f648-4e96-83fa-9cc88e7f728e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:33:59.990797 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 951/1902] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:33:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:33:59.994925 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1296f3-c8d4-41a3-a7da-065f9166bd8c service cinder] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:00.009244 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1296f3-c8d4-41a3-a7da-065f9166bd8c service cinder] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:00.024727 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898de906-edf8-4b53-8678-a78dab74f240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:00.025952 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898de906-edf8-4b53-8678-a78dab74f240 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:00.026776 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898de906-edf8-4b53-8678-a78dab74f240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:00.027547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898de906-edf8-4b53-8678-a78dab74f240 admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:00.029305 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898de906-edf8-4b53-8678-a78dab74f240 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:34:00.030075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898de906-edf8-4b53-8678-a78dab74f240 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:00.032323 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898de906-edf8-4b53-8678-a78dab74f240 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:00.062729 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 952/1903] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:33:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:34:00.067843 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e8f64-f648-4e96-83fa-9cc88e7f728e None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:00.079967 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e8f64-f648-4e96-83fa-9cc88e7f728e None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:00.087998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e313adc7-f3a8-44e8-9121-84ae74cfcd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:00.088731 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313adc7-f3a8-44e8-9121-84ae74cfcd38 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:00.089017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313adc7-f3a8-44e8-9121-84ae74cfcd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:00.090027 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e313adc7-f3a8-44e8-9121-84ae74cfcd38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:00.090802 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e313adc7-f3a8-44e8-9121-84ae74cfcd38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:00.093380 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e313adc7-f3a8-44e8-9121-84ae74cfcd38 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:00.143353 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898de906-edf8-4b53-8678-a78dab74f240 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:00.145312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-898de906-edf8-4b53-8678-a78dab74f240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:00.153615 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 952/1904] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:34:00 2022] POST /identity/v3/projects => generated 389 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:34:00.157662 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e313adc7-f3a8-44e8-9121-84ae74cfcd38 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:00.162342 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898de906-edf8-4b53-8678-a78dab74f240 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:00.248310 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050383f8-f3ef-4d0d-ad39-db2726b13c02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:00.249378 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050383f8-f3ef-4d0d-ad39-db2726b13c02 service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:00.249744 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050383f8-f3ef-4d0d-ad39-db2726b13c02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:00.250101 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050383f8-f3ef-4d0d-ad39-db2726b13c02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:00.252142 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-050383f8-f3ef-4d0d-ad39-db2726b13c02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:00.273204 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e313adc7-f3a8-44e8-9121-84ae74cfcd38 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:00.279779 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af4fcc-dc8a-4116-9a8b-41a026dfdb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:00.280535 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af4fcc-dc8a-4116-9a8b-41a026dfdb41 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:00.280882 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af4fcc-dc8a-4116-9a8b-41a026dfdb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:00.281196 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af4fcc-dc8a-4116-9a8b-41a026dfdb41 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:00.281906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37af4fcc-dc8a-4116-9a8b-41a026dfdb41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:00.283871 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37af4fcc-dc8a-4116-9a8b-41a026dfdb41 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:00.337465 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050383f8-f3ef-4d0d-ad39-db2726b13c02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:00.339139 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050383f8-f3ef-4d0d-ad39-db2726b13c02 service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:00.355890 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 953/1906] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:34:00 2022] POST /identity/v3/users => generated 295 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:34:00.360425 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 953/1906] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:34:00 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:34:00.371029 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af4fcc-dc8a-4116-9a8b-41a026dfdb41 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:00.371219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d9307-5297-471d-bc8f-162c22224b28 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:00.371389 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d9307-5297-471d-bc8f-162c22224b28 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:00.371674 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d9307-5297-471d-bc8f-162c22224b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:00.390060 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af4fcc-dc8a-4116-9a8b-41a026dfdb41 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:00.397719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0939edb9-4f59-41d0-a40c-4d3138ba3375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:00.399480 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939edb9-4f59-41d0-a40c-4d3138ba3375 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:00.399966 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939edb9-4f59-41d0-a40c-4d3138ba3375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:00.400719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939edb9-4f59-41d0-a40c-4d3138ba3375 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:00.401644 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0939edb9-4f59-41d0-a40c-4d3138ba3375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:00.403662 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0939edb9-4f59-41d0-a40c-4d3138ba3375 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:00.437141 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-3a3d9307-5297-471d-bc8f-162c22224b28 None None] MFA Rules not processed for user `6490654f4f4543e6a1f51cfaf543176c`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:00.438681 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-3a3d9307-5297-471d-bc8f-162c22224b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:00.444682 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 954/1907] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:34:00.449835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0939edb9-4f59-41d0-a40c-4d3138ba3375 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:00.461304 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0939edb9-4f59-41d0-a40c-4d3138ba3375 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:00.469552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5170f1-ef98-4883-aae3-719fb4578aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:00.470359 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5170f1-ef98-4883-aae3-719fb4578aa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:00.470719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5170f1-ef98-4883-aae3-719fb4578aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:00.471070 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5170f1-ef98-4883-aae3-719fb4578aa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:00.471927 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5170f1-ef98-4883-aae3-719fb4578aa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:00.473705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5170f1-ef98-4883-aae3-719fb4578aa0 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:00.502856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 955/1908] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:34:00.506811 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5170f1-ef98-4883-aae3-719fb4578aa0 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:00.526259 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5170f1-ef98-4883-aae3-719fb4578aa0 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:00.539251 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90cf8f-085f-4014-8860-547da083e8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:00.540497 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90cf8f-085f-4014-8860-547da083e8f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:00.542392 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90cf8f-085f-4014-8860-547da083e8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:00.542982 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90cf8f-085f-4014-8860-547da083e8f2 admin admin] PATH_INFO: `/v3/projects/1f12896827f44210bb7f34c340c44356/users/7ae083cf35f84f3cb4ca8f9b657511f6/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:00.564630 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc90cf8f-085f-4014-8860-547da083e8f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f12896827f44210bb7f34c340c44356, user_id=7ae083cf35f84f3cb4ca8f9b657511f6, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:00.568162 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc90cf8f-085f-4014-8860-547da083e8f2 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:00.588805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 954/1909] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:00.592850 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d9307-5297-471d-bc8f-162c22224b28 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:00.600628 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-3a3d9307-5297-471d-bc8f-162c22224b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:00.607970 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 956/1910] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:34:00 2022] PUT /identity/v3/projects/1f12896827f44210bb7f34c340c44356/users/7ae083cf35f84f3cb4ca8f9b657511f6/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:34:00.619311 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662aae7-8241-46e5-a75f-52da3b43733e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:00.622417 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662aae7-8241-46e5-a75f-52da3b43733e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:00.622417 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4662aae7-8241-46e5-a75f-52da3b43733e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:00.654212 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d9307-5297-471d-bc8f-162c22224b28 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:00.676230 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:DELETE=1,keystone:SELECT=94,keystone:INSERT=5 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:00.695527 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-4662aae7-8241-46e5-a75f-52da3b43733e None None] MFA Rules not processed for user `7ae083cf35f84f3cb4ca8f9b657511f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:00.698035 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-4662aae7-8241-46e5-a75f-52da3b43733e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:00.731694 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:00.781276 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f2921-92e4-4745-9463-c5df19e03594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:00.782331 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f2921-92e4-4745-9463-c5df19e03594 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:00.782800 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f2921-92e4-4745-9463-c5df19e03594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:00.783255 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f2921-92e4-4745-9463-c5df19e03594 admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:00.784575 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f2921-92e4-4745-9463-c5df19e03594 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:00.789027 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f2921-92e4-4745-9463-c5df19e03594 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:00.829560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 955/1911] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:34:00 2022] POST /identity/v3/projects => generated 373 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:34:00.831514 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f2921-92e4-4745-9463-c5df19e03594 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:00.869287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 957/1912] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:34:00.874546 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662aae7-8241-46e5-a75f-52da3b43733e None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:00.895717 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4662aae7-8241-46e5-a75f-52da3b43733e None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:00.969482 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f2921-92e4-4745-9463-c5df19e03594 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:00.982182 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8981348-bf98-48d8-82dd-27df795fe62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:00.983238 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8981348-bf98-48d8-82dd-27df795fe62e admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:00.983526 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8981348-bf98-48d8-82dd-27df795fe62e admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:00.984219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8981348-bf98-48d8-82dd-27df795fe62e admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:00.984554 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8981348-bf98-48d8-82dd-27df795fe62e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:00.986770 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8981348-bf98-48d8-82dd-27df795fe62e admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:01.002496 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b019a1-faf2-4929-875c-35f7de88b037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:01.004135 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b019a1-faf2-4929-875c-35f7de88b037 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:01.004977 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b019a1-faf2-4929-875c-35f7de88b037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:01.005426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b019a1-faf2-4929-875c-35f7de88b037 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:01.006448 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b019a1-faf2-4929-875c-35f7de88b037 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:01.009624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b019a1-faf2-4929-875c-35f7de88b037 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:01.016058 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 956/1913] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:34:00 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:34:01.019874 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8981348-bf98-48d8-82dd-27df795fe62e admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:01.037285 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8981348-bf98-48d8-82dd-27df795fe62e admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:01.045441 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4dd5a4-25c8-4e59-860a-b2411846a4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:01.045994 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 958/1914] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:34:00 2022] POST /identity/v3/projects => generated 389 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:34:01.047473 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4dd5a4-25c8-4e59-860a-b2411846a4da admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:01.047847 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4dd5a4-25c8-4e59-860a-b2411846a4da admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:01.048223 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4dd5a4-25c8-4e59-860a-b2411846a4da admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:01.049039 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4dd5a4-25c8-4e59-860a-b2411846a4da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:01.049656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b019a1-faf2-4929-875c-35f7de88b037 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:01.051583 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4dd5a4-25c8-4e59-860a-b2411846a4da admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:01.067425 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 957/1915] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:34:01.072094 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4dd5a4-25c8-4e59-860a-b2411846a4da admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:01.084371 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4dd5a4-25c8-4e59-860a-b2411846a4da admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:01.162450 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b019a1-faf2-4929-875c-35f7de88b037 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:01.174993 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751688ff-eca1-4e93-814b-bb0a9eebbb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:01.175696 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751688ff-eca1-4e93-814b-bb0a9eebbb87 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:01.175986 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751688ff-eca1-4e93-814b-bb0a9eebbb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:01.177791 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751688ff-eca1-4e93-814b-bb0a9eebbb87 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:01.178136 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751688ff-eca1-4e93-814b-bb0a9eebbb87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:01.184143 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751688ff-eca1-4e93-814b-bb0a9eebbb87 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:01.201437 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6460a-2c9b-4bac-9b76-74024f41d981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:01.202369 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6460a-2c9b-4bac-9b76-74024f41d981 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:01.202657 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6460a-2c9b-4bac-9b76-74024f41d981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:01.202939 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c6460a-2c9b-4bac-9b76-74024f41d981 admin admin] PATH_INFO: `/v3/projects/866958591e4d49b3b2b6402dbd7db49a/users/a2a9d4a7641a42bfabef15189732e059/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:01.236128 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 959/1916] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:34:01 2022] POST /identity/v3/users => generated 301 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:34:01.237668 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751688ff-eca1-4e93-814b-bb0a9eebbb87 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:01.248662 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6460a-2c9b-4bac-9b76-74024f41d981 admin admin] RBAC: Authorizing `identity:create_grant(project_id=866958591e4d49b3b2b6402dbd7db49a, user_id=a2a9d4a7641a42bfabef15189732e059, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:01.252204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c6460a-2c9b-4bac-9b76-74024f41d981 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:01.265225 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751688ff-eca1-4e93-814b-bb0a9eebbb87 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:01.273372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 958/1917] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:34:01 2022] PUT /identity/v3/projects/866958591e4d49b3b2b6402dbd7db49a/users/a2a9d4a7641a42bfabef15189732e059/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:34:01.279221 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6460a-2c9b-4bac-9b76-74024f41d981 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:01.283425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:01.309202 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:01.355446 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548eeb5-52b3-4ae6-b300-d54b960489cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:01.356593 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548eeb5-52b3-4ae6-b300-d54b960489cd admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:01.357087 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548eeb5-52b3-4ae6-b300-d54b960489cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:01.358290 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548eeb5-52b3-4ae6-b300-d54b960489cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:01.359278 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548eeb5-52b3-4ae6-b300-d54b960489cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:01.361296 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548eeb5-52b3-4ae6-b300-d54b960489cd admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:01.375590 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 960/1918] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:34:01.379638 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548eeb5-52b3-4ae6-b300-d54b960489cd admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:01.382645 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c6460a-2c9b-4bac-9b76-74024f41d981 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:01.394934 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548eeb5-52b3-4ae6-b300-d54b960489cd admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:01.397624 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162f90e-b008-437b-825f-a9506b0a4062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:01.399862 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162f90e-b008-437b-825f-a9506b0a4062 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:01.400704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162f90e-b008-437b-825f-a9506b0a4062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:01.403139 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1162f90e-b008-437b-825f-a9506b0a4062 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:01.403838 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc692a5a-ab75-4772-9f6e-d8871388a558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:01.405151 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc692a5a-ab75-4772-9f6e-d8871388a558 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:01.405664 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc692a5a-ab75-4772-9f6e-d8871388a558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:01.406039 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1162f90e-b008-437b-825f-a9506b0a4062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:01.406206 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc692a5a-ab75-4772-9f6e-d8871388a558 admin admin] PATH_INFO: `/v3/projects/c990794d3b094b39bd4340ff9ef90032/users/0000075b24ca4e419f71c835bf02aa7a/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:01.410786 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1162f90e-b008-437b-825f-a9506b0a4062 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:01.425722 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc692a5a-ab75-4772-9f6e-d8871388a558 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c990794d3b094b39bd4340ff9ef90032, user_id=0000075b24ca4e419f71c835bf02aa7a, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:01.427331 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 959/1919] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:34:01.428811 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc692a5a-ab75-4772-9f6e-d8871388a558 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:01.432249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162f90e-b008-437b-825f-a9506b0a4062 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:01.457654 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1162f90e-b008-437b-825f-a9506b0a4062 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:01.476598 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 961/1920] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:34:01 2022] PUT /identity/v3/projects/c990794d3b094b39bd4340ff9ef90032/users/0000075b24ca4e419f71c835bf02aa7a/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:01.481145 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc692a5a-ab75-4772-9f6e-d8871388a558 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:01.625659 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab7afd-d661-4a1b-88f3-ea0e203b5f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:01.629680 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab7afd-d661-4a1b-88f3-ea0e203b5f53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:01.630610 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab7afd-d661-4a1b-88f3-ea0e203b5f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:01.631346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab7afd-d661-4a1b-88f3-ea0e203b5f53 admin admin] PATH_INFO: `/v3/projects/866958591e4d49b3b2b6402dbd7db49a/users/a2a9d4a7641a42bfabef15189732e059/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:01.636955 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab7afd-d661-4a1b-88f3-ea0e203b5f53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=866958591e4d49b3b2b6402dbd7db49a, user_id=a2a9d4a7641a42bfabef15189732e059, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:01.642534 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab7afd-d661-4a1b-88f3-ea0e203b5f53 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:01.670299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 960/1921] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:34:01 2022] PUT /identity/v3/projects/866958591e4d49b3b2b6402dbd7db49a/users/a2a9d4a7641a42bfabef15189732e059/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:01.671361 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc692a5a-ab75-4772-9f6e-d8871388a558 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:01.775381 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4129ba-f77c-4cf3-9530-c2070af7e95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:01.776692 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4129ba-f77c-4cf3-9530-c2070af7e95d admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:01.776692 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4129ba-f77c-4cf3-9530-c2070af7e95d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:01.777070 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4129ba-f77c-4cf3-9530-c2070af7e95d admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:01.777911 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4129ba-f77c-4cf3-9530-c2070af7e95d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:01.782789 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4129ba-f77c-4cf3-9530-c2070af7e95d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:01.820410 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 962/1922] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:34:01.823018 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab7afd-d661-4a1b-88f3-ea0e203b5f53 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:01.838182 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab7afd-d661-4a1b-88f3-ea0e203b5f53 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:01.847203 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b49cb6-75b4-4ade-8e6e-653d9883caa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:01.849267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b49cb6-75b4-4ade-8e6e-653d9883caa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:01.849632 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b49cb6-75b4-4ade-8e6e-653d9883caa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:01.850096 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b49cb6-75b4-4ade-8e6e-653d9883caa3 admin admin] PATH_INFO: `/v3/projects/c990794d3b094b39bd4340ff9ef90032/users/0000075b24ca4e419f71c835bf02aa7a/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:01.854577 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b49cb6-75b4-4ade-8e6e-653d9883caa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c990794d3b094b39bd4340ff9ef90032, user_id=0000075b24ca4e419f71c835bf02aa7a, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:01.857667 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b49cb6-75b4-4ade-8e6e-653d9883caa3 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:01.877494 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 961/1923] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:34:01 2022] PUT /identity/v3/projects/c990794d3b094b39bd4340ff9ef90032/users/0000075b24ca4e419f71c835bf02aa7a/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:34:01.888895 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d90a9-12b9-4cc9-9738-3c9e6bfb47d6 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:01.889356 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d90a9-12b9-4cc9-9738-3c9e6bfb47d6 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:01.889690 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d90a9-12b9-4cc9-9738-3c9e6bfb47d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:01.931362 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-066d90a9-12b9-4cc9-9738-3c9e6bfb47d6 None None] MFA Rules not processed for user `0000075b24ca4e419f71c835bf02aa7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:01.932888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-066d90a9-12b9-4cc9-9738-3c9e6bfb47d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:01.976211 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:02.082641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 963/1924] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:01 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:02.109019 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b49cb6-75b4-4ade-8e6e-653d9883caa3 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:02.110801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-41b49cb6-75b4-4ade-8e6e-653d9883caa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:02.281297 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b49cb6-75b4-4ade-8e6e-653d9883caa3 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:02.296428 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac99da-5f51-47ba-92a5-ccd30d3d693f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:02.297145 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac99da-5f51-47ba-92a5-ccd30d3d693f service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:02.297999 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac99da-5f51-47ba-92a5-ccd30d3d693f service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:02.298931 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac99da-5f51-47ba-92a5-ccd30d3d693f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:02.314642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ac99da-5f51-47ba-92a5-ccd30d3d693f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:02.316206 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ac99da-5f51-47ba-92a5-ccd30d3d693f service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:02.337025 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 962/1925] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:34:02 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:34:02.636343 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d90a9-12b9-4cc9-9738-3c9e6bfb47d6 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:02.762084 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066d90a9-12b9-4cc9-9738-3c9e6bfb47d6 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:02.767429 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535751c-925a-4d85-9ed7-a0c2173b0212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:02.768350 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535751c-925a-4d85-9ed7-a0c2173b0212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:02.768636 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535751c-925a-4d85-9ed7-a0c2173b0212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:02.768914 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535751c-925a-4d85-9ed7-a0c2173b0212 admin admin] PATH_INFO: `/v3/users/7ae083cf35f84f3cb4ca8f9b657511f6` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:02.770125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3535751c-925a-4d85-9ed7-a0c2173b0212 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ae083cf35f84f3cb4ca8f9b657511f6)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:02.772045 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3535751c-925a-4d85-9ed7-a0c2173b0212 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:02.831403 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3535751c-925a-4d85-9ed7-a0c2173b0212 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ae083cf35f84f3cb4ca8f9b657511f6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:02.849352 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3535751c-925a-4d85-9ed7-a0c2173b0212 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ae083cf35f84f3cb4ca8f9b657511f6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:02.872459 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3535751c-925a-4d85-9ed7-a0c2173b0212 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ae083cf35f84f3cb4ca8f9b657511f6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:02.872974 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3535751c-925a-4d85-9ed7-a0c2173b0212 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ae083cf35f84f3cb4ca8f9b657511f6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:02.874174 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3535751c-925a-4d85-9ed7-a0c2173b0212 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ae083cf35f84f3cb4ca8f9b657511f6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:02.902545 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 964/1926] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:34:02 2022] DELETE /identity/v3/users/7ae083cf35f84f3cb4ca8f9b657511f6 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:02.909892 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19440b13-6c36-40ee-86b4-4fb363d692d3 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:02.911242 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19440b13-6c36-40ee-86b4-4fb363d692d3 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:02.911920 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19440b13-6c36-40ee-86b4-4fb363d692d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:02.938553 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-19440b13-6c36-40ee-86b4-4fb363d692d3 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:02.940652 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-19440b13-6c36-40ee-86b4-4fb363d692d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:03.014083 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:03.127661 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 963/1927] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:03.246342 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535751c-925a-4d85-9ed7-a0c2173b0212 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:03.248729 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3535751c-925a-4d85-9ed7-a0c2173b0212 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:03.325890 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:03.368209 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535751c-925a-4d85-9ed7-a0c2173b0212 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:03.374948 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa4eb60-3e36-4794-a19e-f1d1c27a8bec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:03.375637 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4eb60-3e36-4794-a19e-f1d1c27a8bec service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:03.375922 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4eb60-3e36-4794-a19e-f1d1c27a8bec service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:03.376227 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4eb60-3e36-4794-a19e-f1d1c27a8bec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:03.391246 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa4eb60-3e36-4794-a19e-f1d1c27a8bec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:03.393247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa4eb60-3e36-4794-a19e-f1d1c27a8bec service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:03.410257 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 965/1928] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:34:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:34:04.064499 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758c89d-2077-4c8a-840f-9562cebf2a23 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:04.065410 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758c89d-2077-4c8a-840f-9562cebf2a23 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:04.065692 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758c89d-2077-4c8a-840f-9562cebf2a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:04.118301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-c758c89d-2077-4c8a-840f-9562cebf2a23 None None] MFA Rules not processed for user `a2a9d4a7641a42bfabef15189732e059`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:04.119314 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-c758c89d-2077-4c8a-840f-9562cebf2a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:04.338313 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 964/1929] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:04 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:04.347977 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa4eb60-3e36-4794-a19e-f1d1c27a8bec service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:04.350204 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-faa4eb60-3e36-4794-a19e-f1d1c27a8bec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:04.403658 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758c89d-2077-4c8a-840f-9562cebf2a23 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:04.405643 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-c758c89d-2077-4c8a-840f-9562cebf2a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:04.439238 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758c89d-2077-4c8a-840f-9562cebf2a23 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:04.451091 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379c668f-2d4f-4a23-b985-833c93b3f047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:04.451995 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c668f-2d4f-4a23-b985-833c93b3f047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:04.452317 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c668f-2d4f-4a23-b985-833c93b3f047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:04.452604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c668f-2d4f-4a23-b985-833c93b3f047 admin admin] PATH_INFO: `/v3/projects/1f12896827f44210bb7f34c340c44356` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:04.454799 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379c668f-2d4f-4a23-b985-833c93b3f047 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f12896827f44210bb7f34c340c44356)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:04.459550 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379c668f-2d4f-4a23-b985-833c93b3f047 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:04.524470 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa4eb60-3e36-4794-a19e-f1d1c27a8bec service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:04.530596 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af3cd8-3ed0-4906-b445-f4c21652dda4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:04.531281 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af3cd8-3ed0-4906-b445-f4c21652dda4 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:04.531567 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af3cd8-3ed0-4906-b445-f4c21652dda4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:04.531843 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af3cd8-3ed0-4906-b445-f4c21652dda4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:04.533818 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b3af3cd8-3ed0-4906-b445-f4c21652dda4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:04.609310 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-379c668f-2d4f-4a23-b985-833c93b3f047 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f12896827f44210bb7f34c340c44356', 'request_id': 'req-379c668f-2d4f-4a23-b985-833c93b3f047'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:04.653874 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-379c668f-2d4f-4a23-b985-833c93b3f047 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f12896827f44210bb7f34c340c44356', 'request_id': 'req-379c668f-2d4f-4a23-b985-833c93b3f047'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:04.671763 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-379c668f-2d4f-4a23-b985-833c93b3f047 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f12896827f44210bb7f34c340c44356', 'request_id': 'req-379c668f-2d4f-4a23-b985-833c93b3f047'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:04.698378 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3af3cd8-3ed0-4906-b445-f4c21652dda4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:04.699912 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3af3cd8-3ed0-4906-b445-f4c21652dda4 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:04.713750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-379c668f-2d4f-4a23-b985-833c93b3f047 admin admin] The token cache is being invalidate because project 1f12896827f44210bb7f34c340c44356 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:34:04.714137 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-379c668f-2d4f-4a23-b985-833c93b3f047 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:04.715031 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 966/1930] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:34:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:34:04.715645 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-379c668f-2d4f-4a23-b985-833c93b3f047 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:04.720348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 965/1931] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:34:04 2022] DELETE /identity/v3/projects/1f12896827f44210bb7f34c340c44356 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:04.721772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af3cd8-3ed0-4906-b445-f4c21652dda4 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:04.734819 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af3cd8-3ed0-4906-b445-f4c21652dda4 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:04.818466 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b404b-26a3-48f5-87c9-301197d5b9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:04.819367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b404b-26a3-48f5-87c9-301197d5b9e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:04.819648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b404b-26a3-48f5-87c9-301197d5b9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:04.819993 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b404b-26a3-48f5-87c9-301197d5b9e5 admin admin] PATH_INFO: `/v3/users/0000075b24ca4e419f71c835bf02aa7a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:04.821606 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34b404b-26a3-48f5-87c9-301197d5b9e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0000075b24ca4e419f71c835bf02aa7a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:04.823643 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34b404b-26a3-48f5-87c9-301197d5b9e5 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:04.883334 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:04.906843 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c34b404b-26a3-48f5-87c9-301197d5b9e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0000075b24ca4e419f71c835bf02aa7a'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:04.935778 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c34b404b-26a3-48f5-87c9-301197d5b9e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0000075b24ca4e419f71c835bf02aa7a'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:04.962074 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c34b404b-26a3-48f5-87c9-301197d5b9e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0000075b24ca4e419f71c835bf02aa7a'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:04.963338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c34b404b-26a3-48f5-87c9-301197d5b9e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0000075b24ca4e419f71c835bf02aa7a'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:04.964445 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c34b404b-26a3-48f5-87c9-301197d5b9e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0000075b24ca4e419f71c835bf02aa7a'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:04.999286 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 967/1932] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:34:04 2022] DELETE /identity/v3/users/0000075b24ca4e419f71c835bf02aa7a => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:05.798099 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379c668f-2d4f-4a23-b985-833c93b3f047 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:05.800629 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-379c668f-2d4f-4a23-b985-833c93b3f047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:05.824548 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379c668f-2d4f-4a23-b985-833c93b3f047 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:05.879662 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:05.908997 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90802d9-96a7-4f6c-b395-de0f62154c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:05.909885 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90802d9-96a7-4f6c-b395-de0f62154c52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:05.910172 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90802d9-96a7-4f6c-b395-de0f62154c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:05.910451 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90802d9-96a7-4f6c-b395-de0f62154c52 admin admin] PATH_INFO: `/v3/projects/c990794d3b094b39bd4340ff9ef90032` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:05.911768 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90802d9-96a7-4f6c-b395-de0f62154c52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c990794d3b094b39bd4340ff9ef90032)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:05.915148 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90802d9-96a7-4f6c-b395-de0f62154c52 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:06.033999 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-d90802d9-96a7-4f6c-b395-de0f62154c52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c990794d3b094b39bd4340ff9ef90032', 'request_id': 'req-d90802d9-96a7-4f6c-b395-de0f62154c52'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:06.042949 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-d90802d9-96a7-4f6c-b395-de0f62154c52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c990794d3b094b39bd4340ff9ef90032', 'request_id': 'req-d90802d9-96a7-4f6c-b395-de0f62154c52'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:06.049546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-d90802d9-96a7-4f6c-b395-de0f62154c52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c990794d3b094b39bd4340ff9ef90032', 'request_id': 'req-d90802d9-96a7-4f6c-b395-de0f62154c52'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:06.061328 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-d90802d9-96a7-4f6c-b395-de0f62154c52 admin admin] The token cache is being invalidate because project c990794d3b094b39bd4340ff9ef90032 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:34:06.061619 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-d90802d9-96a7-4f6c-b395-de0f62154c52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:06.062299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-d90802d9-96a7-4f6c-b395-de0f62154c52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:06.063977 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 966/1933] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:34:05 2022] DELETE /identity/v3/projects/c990794d3b094b39bd4340ff9ef90032 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:06.069084 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f5e80-f764-4a63-b554-834211a2e4b8 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:06.069555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f5e80-f764-4a63-b554-834211a2e4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:06.069663 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f5e80-f764-4a63-b554-834211a2e4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:06.092361 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-9e8f5e80-f764-4a63-b554-834211a2e4b8 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:06.093786 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-9e8f5e80-f764-4a63-b554-834211a2e4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:06.186541 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 968/1934] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:34:06.194052 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fef3fb-e152-4579-a7ff-d42ad7868028 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:06.194363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fef3fb-e152-4579-a7ff-d42ad7868028 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:06.195432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fef3fb-e152-4579-a7ff-d42ad7868028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:06.220762 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-c0fef3fb-e152-4579-a7ff-d42ad7868028 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:06.222036 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-c0fef3fb-e152-4579-a7ff-d42ad7868028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:06.240464 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 967/1935] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:06.245142 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f5e80-f764-4a63-b554-834211a2e4b8 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:06.256858 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f5e80-f764-4a63-b554-834211a2e4b8 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:06.262699 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1972c-1b2e-4999-afa9-d4df799940a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:06.263549 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1972c-1b2e-4999-afa9-d4df799940a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:06.263967 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1972c-1b2e-4999-afa9-d4df799940a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:06.264590 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1972c-1b2e-4999-afa9-d4df799940a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:06.265332 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e1972c-1b2e-4999-afa9-d4df799940a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:34:06.265780 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e1972c-1b2e-4999-afa9-d4df799940a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:06.267610 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e1972c-1b2e-4999-afa9-d4df799940a0 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:06.282020 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 969/1936] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:34:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:34:06.286763 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fef3fb-e152-4579-a7ff-d42ad7868028 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:06.296381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fef3fb-e152-4579-a7ff-d42ad7868028 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:06.301018 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90592379-58bc-4d1b-86c5-1490790467b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:06.301676 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90592379-58bc-4d1b-86c5-1490790467b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:06.301964 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90592379-58bc-4d1b-86c5-1490790467b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:06.302244 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90592379-58bc-4d1b-86c5-1490790467b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:06.302935 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90592379-58bc-4d1b-86c5-1490790467b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:06.304911 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90592379-58bc-4d1b-86c5-1490790467b3 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:06.339560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 968/1937] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:34:06 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:34:06.343676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1972c-1b2e-4999-afa9-d4df799940a0 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:06.354470 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1972c-1b2e-4999-afa9-d4df799940a0 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:06.374980 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90592379-58bc-4d1b-86c5-1490790467b3 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:06.380327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-90592379-58bc-4d1b-86c5-1490790467b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:06.427964 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc10cb8-c9e0-4069-a130-433723ff559b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:06.428709 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc10cb8-c9e0-4069-a130-433723ff559b admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:06.428995 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc10cb8-c9e0-4069-a130-433723ff559b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:06.429410 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc10cb8-c9e0-4069-a130-433723ff559b admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:06.430892 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc10cb8-c9e0-4069-a130-433723ff559b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:06.433110 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc10cb8-c9e0-4069-a130-433723ff559b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:06.443446 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90592379-58bc-4d1b-86c5-1490790467b3 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:06.449661 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0ba4b-80fe-4c2c-974c-9ece3cb1fa18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:06.451958 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0ba4b-80fe-4c2c-974c-9ece3cb1fa18 service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:06.451958 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0ba4b-80fe-4c2c-974c-9ece3cb1fa18 service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:06.451958 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0ba4b-80fe-4c2c-974c-9ece3cb1fa18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:06.453544 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-9dc0ba4b-80fe-4c2c-974c-9ece3cb1fa18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:06.455452 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 970/1938] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:34:06 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:34:06.459660 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc10cb8-c9e0-4069-a130-433723ff559b admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:06.469595 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc10cb8-c9e0-4069-a130-433723ff559b admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:06.474756 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1669d3-c945-4808-995e-c0a56b9f7106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:06.475438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1669d3-c945-4808-995e-c0a56b9f7106 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:06.475725 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1669d3-c945-4808-995e-c0a56b9f7106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:06.476004 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1669d3-c945-4808-995e-c0a56b9f7106 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:06.476770 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1669d3-c945-4808-995e-c0a56b9f7106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:06.478417 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1669d3-c945-4808-995e-c0a56b9f7106 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:06.493135 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 971/1939] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:34:06.496732 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1669d3-c945-4808-995e-c0a56b9f7106 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:06.508897 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1669d3-c945-4808-995e-c0a56b9f7106 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:06.513579 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c1433-a538-49c9-b0ea-a60dd0507976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:06.514258 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c1433-a538-49c9-b0ea-a60dd0507976 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:06.514545 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c1433-a538-49c9-b0ea-a60dd0507976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:06.514826 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c1433-a538-49c9-b0ea-a60dd0507976 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:06.515551 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c1433-a538-49c9-b0ea-a60dd0507976 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:06.517181 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c1433-a538-49c9-b0ea-a60dd0507976 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:06.523123 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:06.525875 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc0ba4b-80fe-4c2c-974c-9ece3cb1fa18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:06.527498 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc0ba4b-80fe-4c2c-974c-9ece3cb1fa18 service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:06.532736 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 972/1940] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:34:06.534188 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c1433-a538-49c9-b0ea-a60dd0507976 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:06.542443 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:06.545216 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 969/1941] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:34:06 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:34:06.547209 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c1433-a538-49c9-b0ea-a60dd0507976 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:06.553825 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce9e7a3-15e4-4de9-9f1c-592f0feac336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:06.554774 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9e7a3-15e4-4de9-9f1c-592f0feac336 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:06.556396 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9e7a3-15e4-4de9-9f1c-592f0feac336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:06.556968 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9e7a3-15e4-4de9-9f1c-592f0feac336 admin admin] PATH_INFO: `/v3/projects/60b0caf397e04eb88297de46e7f556e0/users/884b5ede502c48459d2770dedcb7aacc/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:06.581071 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce9e7a3-15e4-4de9-9f1c-592f0feac336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b0caf397e04eb88297de46e7f556e0, user_id=884b5ede502c48459d2770dedcb7aacc, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:06.584384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce9e7a3-15e4-4de9-9f1c-592f0feac336 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:06.608953 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 973/1942] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:34:06 2022] PUT /identity/v3/projects/60b0caf397e04eb88297de46e7f556e0/users/884b5ede502c48459d2770dedcb7aacc/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:06.620310 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3536b4-982e-457a-9d7a-2fc39e8f92e8 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:06.620644 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3536b4-982e-457a-9d7a-2fc39e8f92e8 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:06.620896 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3536b4-982e-457a-9d7a-2fc39e8f92e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:06.678492 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-0d3536b4-982e-457a-9d7a-2fc39e8f92e8 None None] MFA Rules not processed for user `884b5ede502c48459d2770dedcb7aacc`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:06.681673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-0d3536b4-982e-457a-9d7a-2fc39e8f92e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:06.799086 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 970/1943] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:34:06.810390 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce9e7a3-15e4-4de9-9f1c-592f0feac336 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:06.812115 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-4ce9e7a3-15e4-4de9-9f1c-592f0feac336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:06.903105 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3536b4-982e-457a-9d7a-2fc39e8f92e8 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:06.905593 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-0d3536b4-982e-457a-9d7a-2fc39e8f92e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:06.982621 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce9e7a3-15e4-4de9-9f1c-592f0feac336 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:07.012379 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c60d0-a623-4669-9621-0f7e0a9a237f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2483a351f5e943ad8c872a98a4717b0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:07.013123 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c60d0-a623-4669-9621-0f7e0a9a237f service glance] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:07.013408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c60d0-a623-4669-9621-0f7e0a9a237f service glance] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:07.014194 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c60d0-a623-4669-9621-0f7e0a9a237f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:07.033259 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545c60d0-a623-4669-9621-0f7e0a9a237f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:07.035009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545c60d0-a623-4669-9621-0f7e0a9a237f service glance] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:07.056633 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 974/1944] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:34:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:34:07.099964 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3536b4-982e-457a-9d7a-2fc39e8f92e8 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:07.106038 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d5a5f-a483-499a-b0ab-d8ec517d51f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:07.106930 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d5a5f-a483-499a-b0ab-d8ec517d51f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:07.108006 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d5a5f-a483-499a-b0ab-d8ec517d51f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:07.108290 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d5a5f-a483-499a-b0ab-d8ec517d51f8 admin admin] PATH_INFO: `/v3/users/600c873e0d1645268e49ec8144fc913c` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:07.109544 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112d5a5f-a483-499a-b0ab-d8ec517d51f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=600c873e0d1645268e49ec8144fc913c)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:07.111532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112d5a5f-a483-499a-b0ab-d8ec517d51f8 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:07.191980 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-112d5a5f-a483-499a-b0ab-d8ec517d51f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '600c873e0d1645268e49ec8144fc913c'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:07.192763 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-112d5a5f-a483-499a-b0ab-d8ec517d51f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '600c873e0d1645268e49ec8144fc913c'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:07.204381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-112d5a5f-a483-499a-b0ab-d8ec517d51f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '600c873e0d1645268e49ec8144fc913c'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:07.224293 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-112d5a5f-a483-499a-b0ab-d8ec517d51f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '600c873e0d1645268e49ec8144fc913c'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:07.238529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-112d5a5f-a483-499a-b0ab-d8ec517d51f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '600c873e0d1645268e49ec8144fc913c'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:07.248813 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 971/1945] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:34:06 2022] DELETE /identity/v3/users/600c873e0d1645268e49ec8144fc913c => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:07.253611 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e27db-6570-4c92-ada4-c64f84dc9f42 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:07.254111 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e27db-6570-4c92-ada4-c64f84dc9f42 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:07.254215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e27db-6570-4c92-ada4-c64f84dc9f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:07.280603 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-5c1e27db-6570-4c92-ada4-c64f84dc9f42 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:07.282260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5c1e27db-6570-4c92-ada4-c64f84dc9f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:07.385488 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 975/1946] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:07.404547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d5a5f-a483-499a-b0ab-d8ec517d51f8 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:07.406046 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-112d5a5f-a483-499a-b0ab-d8ec517d51f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:07.427183 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:07.534256 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d5a5f-a483-499a-b0ab-d8ec517d51f8 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:07.538794 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02481860-004f-4f90-b700-a2525bf7fd40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:07.540859 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02481860-004f-4f90-b700-a2525bf7fd40 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:07.541498 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02481860-004f-4f90-b700-a2525bf7fd40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:07.541992 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02481860-004f-4f90-b700-a2525bf7fd40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:07.556485 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02481860-004f-4f90-b700-a2525bf7fd40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:07.558570 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02481860-004f-4f90-b700-a2525bf7fd40 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:07.572389 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 972/1947] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:34:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:34:08.231236 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1e27db-6570-4c92-ada4-c64f84dc9f42 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:08.233373 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5c1e27db-6570-4c92-ada4-c64f84dc9f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:08.254457 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1e27db-6570-4c92-ada4-c64f84dc9f42 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:08.263706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a1ca7-7fe5-4cd8-820d-06b6c4daf6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:08.266417 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a1ca7-7fe5-4cd8-820d-06b6c4daf6b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:08.266695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a1ca7-7fe5-4cd8-820d-06b6c4daf6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:08.266996 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a1ca7-7fe5-4cd8-820d-06b6c4daf6b6 admin admin] PATH_INFO: `/v3/projects/7cb4a58a707f4ddca6930fd35a34c30c` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:08.270223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a1ca7-7fe5-4cd8-820d-06b6c4daf6b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cb4a58a707f4ddca6930fd35a34c30c)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:08.272368 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a1ca7-7fe5-4cd8-820d-06b6c4daf6b6 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:08.369961 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-794a1ca7-7fe5-4cd8-820d-06b6c4daf6b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cb4a58a707f4ddca6930fd35a34c30c', 'request_id': 'req-794a1ca7-7fe5-4cd8-820d-06b6c4daf6b6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:08.376976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-794a1ca7-7fe5-4cd8-820d-06b6c4daf6b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cb4a58a707f4ddca6930fd35a34c30c', 'request_id': 'req-794a1ca7-7fe5-4cd8-820d-06b6c4daf6b6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:08.389884 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-794a1ca7-7fe5-4cd8-820d-06b6c4daf6b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cb4a58a707f4ddca6930fd35a34c30c', 'request_id': 'req-794a1ca7-7fe5-4cd8-820d-06b6c4daf6b6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:08.435908 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-794a1ca7-7fe5-4cd8-820d-06b6c4daf6b6 admin admin] The token cache is being invalidate because project 7cb4a58a707f4ddca6930fd35a34c30c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:34:08.436263 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-794a1ca7-7fe5-4cd8-820d-06b6c4daf6b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:08.437113 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-794a1ca7-7fe5-4cd8-820d-06b6c4daf6b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:08.440776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 976/1948] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:34:08 2022] DELETE /identity/v3/projects/7cb4a58a707f4ddca6930fd35a34c30c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:08.445151 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02481860-004f-4f90-b700-a2525bf7fd40 service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:08.466436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02481860-004f-4f90-b700-a2525bf7fd40 service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:08.578384 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde7f1c-4b5b-4c2b-b9dd-d39a9b383a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:08.579337 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde7f1c-4b5b-4c2b-b9dd-d39a9b383a47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:08.579627 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde7f1c-4b5b-4c2b-b9dd-d39a9b383a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:08.579956 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde7f1c-4b5b-4c2b-b9dd-d39a9b383a47 admin admin] PATH_INFO: `/v3/users/d8e336fdfc1146b6b960ec7fb50f0ce3` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:08.581235 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde7f1c-4b5b-4c2b-b9dd-d39a9b383a47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8e336fdfc1146b6b960ec7fb50f0ce3)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:08.584116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde7f1c-4b5b-4c2b-b9dd-d39a9b383a47 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:08.660048 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-7dde7f1c-4b5b-4c2b-b9dd-d39a9b383a47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8e336fdfc1146b6b960ec7fb50f0ce3'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:08.660885 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-7dde7f1c-4b5b-4c2b-b9dd-d39a9b383a47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8e336fdfc1146b6b960ec7fb50f0ce3'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:08.675794 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-7dde7f1c-4b5b-4c2b-b9dd-d39a9b383a47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8e336fdfc1146b6b960ec7fb50f0ce3'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:08.705560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-7dde7f1c-4b5b-4c2b-b9dd-d39a9b383a47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8e336fdfc1146b6b960ec7fb50f0ce3'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:08.714914 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-7dde7f1c-4b5b-4c2b-b9dd-d39a9b383a47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8e336fdfc1146b6b960ec7fb50f0ce3'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:08.725250 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 973/1949] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:34:08 2022] DELETE /identity/v3/users/d8e336fdfc1146b6b960ec7fb50f0ce3 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:09.429795 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a1ca7-7fe5-4cd8-820d-06b6c4daf6b6 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:09.432270 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-794a1ca7-7fe5-4cd8-820d-06b6c4daf6b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:09.454544 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a1ca7-7fe5-4cd8-820d-06b6c4daf6b6 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:09.460978 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:09.545585 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e769d127-8e04-469e-8d6f-e97c2be34c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:09.546688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769d127-8e04-469e-8d6f-e97c2be34c11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:09.547310 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769d127-8e04-469e-8d6f-e97c2be34c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:09.547686 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769d127-8e04-469e-8d6f-e97c2be34c11 admin admin] PATH_INFO: `/v3/projects/5ecc73eaacc4455e93cfba924de6e3a7` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:09.552609 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e769d127-8e04-469e-8d6f-e97c2be34c11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ecc73eaacc4455e93cfba924de6e3a7)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:09.554796 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e769d127-8e04-469e-8d6f-e97c2be34c11 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:09.688982 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e769d127-8e04-469e-8d6f-e97c2be34c11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ecc73eaacc4455e93cfba924de6e3a7', 'request_id': 'req-e769d127-8e04-469e-8d6f-e97c2be34c11'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:09.723980 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e769d127-8e04-469e-8d6f-e97c2be34c11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ecc73eaacc4455e93cfba924de6e3a7', 'request_id': 'req-e769d127-8e04-469e-8d6f-e97c2be34c11'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:09.748886 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e769d127-8e04-469e-8d6f-e97c2be34c11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ecc73eaacc4455e93cfba924de6e3a7', 'request_id': 'req-e769d127-8e04-469e-8d6f-e97c2be34c11'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:09.784835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e769d127-8e04-469e-8d6f-e97c2be34c11 admin admin] The token cache is being invalidate because project 5ecc73eaacc4455e93cfba924de6e3a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:34:09.790848 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e769d127-8e04-469e-8d6f-e97c2be34c11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:09.792077 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e769d127-8e04-469e-8d6f-e97c2be34c11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:09.802913 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 977/1950] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:34:09 2022] DELETE /identity/v3/projects/5ecc73eaacc4455e93cfba924de6e3a7 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:09.808392 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0e900-ea35-4dc4-b98b-ed269d87f1ef None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:09.809155 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0e900-ea35-4dc4-b98b-ed269d87f1ef None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:09.809155 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0e900-ea35-4dc4-b98b-ed269d87f1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:09.853508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-15d0e900-ea35-4dc4-b98b-ed269d87f1ef None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:09.857153 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-15d0e900-ea35-4dc4-b98b-ed269d87f1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:09.928621 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:10.043976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 974/1951] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:10.046079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5edab-43ea-496f-ab6f-7739988b1aaf None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:10.046929 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5edab-43ea-496f-ab6f-7739988b1aaf None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:10.046929 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5edab-43ea-496f-ab6f-7739988b1aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:10.086508 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-e0b5edab-43ea-496f-ab6f-7739988b1aaf None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:10.090245 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e0b5edab-43ea-496f-ab6f-7739988b1aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:10.108712 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 978/1952] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:10.113669 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d0e900-ea35-4dc4-b98b-ed269d87f1ef None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:10.130528 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d0e900-ea35-4dc4-b98b-ed269d87f1ef None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:10.140037 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c26cde-1fb0-4af9-9ea9-9f250f1c0042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:10.140856 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c26cde-1fb0-4af9-9ea9-9f250f1c0042 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:10.141209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c26cde-1fb0-4af9-9ea9-9f250f1c0042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:10.142072 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c26cde-1fb0-4af9-9ea9-9f250f1c0042 admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:10.142907 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c26cde-1fb0-4af9-9ea9-9f250f1c0042 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:34:10.143286 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c26cde-1fb0-4af9-9ea9-9f250f1c0042 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:10.146066 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c26cde-1fb0-4af9-9ea9-9f250f1c0042 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:10.166216 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 975/1953] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:34:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:34:10.167182 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5edab-43ea-496f-ab6f-7739988b1aaf None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:10.183402 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5edab-43ea-496f-ab6f-7739988b1aaf None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:10.191732 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4333e713-648c-4e88-a936-0104c3408eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:10.193086 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4333e713-648c-4e88-a936-0104c3408eb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:10.194240 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4333e713-648c-4e88-a936-0104c3408eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:10.194763 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4333e713-648c-4e88-a936-0104c3408eb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:10.196039 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4333e713-648c-4e88-a936-0104c3408eb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:10.198641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4333e713-648c-4e88-a936-0104c3408eb7 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:10.254803 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 979/1954] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:34:10 2022] POST /identity/v3/projects => generated 383 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:34:10.256374 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c26cde-1fb0-4af9-9ea9-9f250f1c0042 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:10.279890 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c26cde-1fb0-4af9-9ea9-9f250f1c0042 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:10.396764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45748b36-7e57-4f47-a811-3b63ad9f636f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:10.397653 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45748b36-7e57-4f47-a811-3b63ad9f636f admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:10.397855 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45748b36-7e57-4f47-a811-3b63ad9f636f admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:10.398214 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45748b36-7e57-4f47-a811-3b63ad9f636f admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:10.403554 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45748b36-7e57-4f47-a811-3b63ad9f636f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:10.408973 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45748b36-7e57-4f47-a811-3b63ad9f636f admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:10.440059 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 976/1955] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:34:10 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:34:10.444839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4333e713-648c-4e88-a936-0104c3408eb7 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:10.456510 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4333e713-648c-4e88-a936-0104c3408eb7 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:10.467335 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55036639-cd6e-4dfe-8526-32fa3ebd7b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:10.468024 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55036639-cd6e-4dfe-8526-32fa3ebd7b8b admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:10.468396 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55036639-cd6e-4dfe-8526-32fa3ebd7b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:10.468681 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55036639-cd6e-4dfe-8526-32fa3ebd7b8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:10.469426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55036639-cd6e-4dfe-8526-32fa3ebd7b8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:10.471078 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55036639-cd6e-4dfe-8526-32fa3ebd7b8b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:10.486530 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 980/1956] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:34:10.489993 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45748b36-7e57-4f47-a811-3b63ad9f636f admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:10.504189 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45748b36-7e57-4f47-a811-3b63ad9f636f admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:10.511969 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d4ea6-4bc6-4042-97c7-414617f7b1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:10.513208 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d4ea6-4bc6-4042-97c7-414617f7b1cd admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:10.513208 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d4ea6-4bc6-4042-97c7-414617f7b1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:10.513563 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d4ea6-4bc6-4042-97c7-414617f7b1cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:10.514158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d4ea6-4bc6-4042-97c7-414617f7b1cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:10.516236 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d4ea6-4bc6-4042-97c7-414617f7b1cd admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:10.536827 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 977/1957] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:34:10.541793 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55036639-cd6e-4dfe-8526-32fa3ebd7b8b admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:10.557123 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55036639-cd6e-4dfe-8526-32fa3ebd7b8b admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:10.562317 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31707c3-d4cd-4b9a-854c-39908a585d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:10.563354 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31707c3-d4cd-4b9a-854c-39908a585d96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:10.563710 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31707c3-d4cd-4b9a-854c-39908a585d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:10.564067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31707c3-d4cd-4b9a-854c-39908a585d96 admin admin] PATH_INFO: `/v3/projects/60c2bf1c616f4e0083f6f42c0d9721fb/users/94b9a4d32a034ad79852abb6a197d3d7/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:10.591569 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31707c3-d4cd-4b9a-854c-39908a585d96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60c2bf1c616f4e0083f6f42c0d9721fb, user_id=94b9a4d32a034ad79852abb6a197d3d7, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:10.594470 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31707c3-d4cd-4b9a-854c-39908a585d96 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:10.610915 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 981/1958] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:34:10 2022] PUT /identity/v3/projects/60c2bf1c616f4e0083f6f42c0d9721fb/users/94b9a4d32a034ad79852abb6a197d3d7/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:10.622983 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf3c7b-940f-4b3a-a09f-1938cdcf8ee7 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:10.624599 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf3c7b-940f-4b3a-a09f-1938cdcf8ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:10.624911 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf3c7b-940f-4b3a-a09f-1938cdcf8ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:10.663989 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-0cdf3c7b-940f-4b3a-a09f-1938cdcf8ee7 None None] MFA Rules not processed for user `94b9a4d32a034ad79852abb6a197d3d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:10.666511 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-0cdf3c7b-940f-4b3a-a09f-1938cdcf8ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:10.765203 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 978/1959] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:10.777218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31707c3-d4cd-4b9a-854c-39908a585d96 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:10.779208 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e31707c3-d4cd-4b9a-854c-39908a585d96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:10.829539 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:10.860697 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31707c3-d4cd-4b9a-854c-39908a585d96 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:10.867211 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe467b16-b737-4015-bab3-cf7190fb9e14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:10.868460 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe467b16-b737-4015-bab3-cf7190fb9e14 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:10.868460 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe467b16-b737-4015-bab3-cf7190fb9e14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:10.868780 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe467b16-b737-4015-bab3-cf7190fb9e14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:10.880221 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe467b16-b737-4015-bab3-cf7190fb9e14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:10.881696 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe467b16-b737-4015-bab3-cf7190fb9e14 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:10.906525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 982/1960] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:34:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:34:13.722593 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea168e27-7444-4222-8859-9c158cc336db None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:13.723249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea168e27-7444-4222-8859-9c158cc336db None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:13.723249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea168e27-7444-4222-8859-9c158cc336db None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:13.724667 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 979/1961] 10.222.0.16 () {54 vars in 921 bytes} [Mon May 23 12:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:34:13.732114 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1cb567-f095-4cf3-b721-4a32e408ba92 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:13.733206 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1cb567-f095-4cf3-b721-4a32e408ba92 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:13.733642 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1cb567-f095-4cf3-b721-4a32e408ba92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:13.755490 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-ab1cb567-f095-4cf3-b721-4a32e408ba92 None None] MFA Rules not processed for user `ebc0f137e0d24dd2a460689bc17a1b98`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:13.756758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ab1cb567-f095-4cf3-b721-4a32e408ba92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:13.797852 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea168e27-7444-4222-8859-9c158cc336db None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:13.799700 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-ea168e27-7444-4222-8859-9c158cc336db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:13.845808 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:13.860095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 983/1962] 10.222.0.16 () {58 vars in 1018 bytes} [Mon May 23 12:34:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:13.873735 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1cb567-f095-4cf3-b721-4a32e408ba92 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:13.875342 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ab1cb567-f095-4cf3-b721-4a32e408ba92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:13.898295 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1cb567-f095-4cf3-b721-4a32e408ba92 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:13.904371 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b7429-d1ab-4312-9219-dd2734b3f92e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc0f137e0d24dd2a460689bc17a1b98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:13.905156 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b7429-d1ab-4312-9219-dd2734b3f92e service placement] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:13.905569 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b7429-d1ab-4312-9219-dd2734b3f92e service placement] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:13.905923 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b7429-d1ab-4312-9219-dd2734b3f92e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:13.916841 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b7429-d1ab-4312-9219-dd2734b3f92e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:13.918602 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b7429-d1ab-4312-9219-dd2734b3f92e service placement] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:13.929028 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea168e27-7444-4222-8859-9c158cc336db None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:13.931338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 984/1963] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:34:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:34:13.939804 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed4852-6340-4431-b685-e8ee3d67ebfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:13.942377 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed4852-6340-4431-b685-e8ee3d67ebfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:13.942377 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed4852-6340-4431-b685-e8ee3d67ebfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:13.942377 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed4852-6340-4431-b685-e8ee3d67ebfc admin admin] PATH_INFO: `/v3/users/f2936fe7026046aa9c43f1b1b0cb1bfd` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:13.943664 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed4852-6340-4431-b685-e8ee3d67ebfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2936fe7026046aa9c43f1b1b0cb1bfd)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:13.950590 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed4852-6340-4431-b685-e8ee3d67ebfc admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:14.018019 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-ebed4852-6340-4431-b685-e8ee3d67ebfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2936fe7026046aa9c43f1b1b0cb1bfd'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:14.018786 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-ebed4852-6340-4431-b685-e8ee3d67ebfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2936fe7026046aa9c43f1b1b0cb1bfd'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:14.027537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-ebed4852-6340-4431-b685-e8ee3d67ebfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2936fe7026046aa9c43f1b1b0cb1bfd'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:14.043415 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-ebed4852-6340-4431-b685-e8ee3d67ebfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2936fe7026046aa9c43f1b1b0cb1bfd'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:14.055615 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-ebed4852-6340-4431-b685-e8ee3d67ebfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2936fe7026046aa9c43f1b1b0cb1bfd'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:14.062900 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 980/1964] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:34:13 2022] DELETE /identity/v3/users/f2936fe7026046aa9c43f1b1b0cb1bfd => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:34:14.279536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b7429-d1ab-4312-9219-dd2734b3f92e service placement] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:14.294204 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b7429-d1ab-4312-9219-dd2734b3f92e service placement] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:14.378350 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f73dc-2b1c-41ce-ab6b-9ff254c41597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:14.379265 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f73dc-2b1c-41ce-ab6b-9ff254c41597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:14.379551 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f73dc-2b1c-41ce-ab6b-9ff254c41597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:14.379831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f73dc-2b1c-41ce-ab6b-9ff254c41597 admin admin] PATH_INFO: `/v3/projects/1f94aad6b61d49da817d70d2b91a7594` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:14.382961 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f73dc-2b1c-41ce-ab6b-9ff254c41597 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f94aad6b61d49da817d70d2b91a7594)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:14.385021 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f73dc-2b1c-41ce-ab6b-9ff254c41597 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:14.443453 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c91f73dc-2b1c-41ce-ab6b-9ff254c41597 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f94aad6b61d49da817d70d2b91a7594', 'request_id': 'req-c91f73dc-2b1c-41ce-ab6b-9ff254c41597'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:14.453855 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c91f73dc-2b1c-41ce-ab6b-9ff254c41597 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f94aad6b61d49da817d70d2b91a7594', 'request_id': 'req-c91f73dc-2b1c-41ce-ab6b-9ff254c41597'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:14.456645 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:14.462082 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c91f73dc-2b1c-41ce-ab6b-9ff254c41597 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f94aad6b61d49da817d70d2b91a7594', 'request_id': 'req-c91f73dc-2b1c-41ce-ab6b-9ff254c41597'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:14.479873 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c91f73dc-2b1c-41ce-ab6b-9ff254c41597 admin admin] The token cache is being invalidate because project 1f94aad6b61d49da817d70d2b91a7594 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:34:14.480662 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c91f73dc-2b1c-41ce-ab6b-9ff254c41597 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:14.481692 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c91f73dc-2b1c-41ce-ab6b-9ff254c41597 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:14.484267 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 985/1965] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:34:14 2022] DELETE /identity/v3/projects/1f94aad6b61d49da817d70d2b91a7594 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:14.490343 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab6ff6-ecca-4168-ab73-598cc60a056b None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:14.490762 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab6ff6-ecca-4168-ab73-598cc60a056b None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:14.491109 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab6ff6-ecca-4168-ab73-598cc60a056b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:14.513852 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-dfab6ff6-ecca-4168-ab73-598cc60a056b None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:14.515088 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-dfab6ff6-ecca-4168-ab73-598cc60a056b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:14.581888 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 981/1966] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:14.590206 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632ed43-2b18-456b-9034-c4027c80cfe5 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:14.590496 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632ed43-2b18-456b-9034-c4027c80cfe5 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:14.590779 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632ed43-2b18-456b-9034-c4027c80cfe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:14.610916 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-2632ed43-2b18-456b-9034-c4027c80cfe5 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:14.612137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-2632ed43-2b18-456b-9034-c4027c80cfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:14.620488 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 986/1967] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:14.624855 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab6ff6-ecca-4168-ab73-598cc60a056b None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:14.635437 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab6ff6-ecca-4168-ab73-598cc60a056b None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:14.640093 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac421eb3-6c8b-4b1a-b36f-2689488fd4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:14.640783 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac421eb3-6c8b-4b1a-b36f-2689488fd4fb admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:14.641069 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac421eb3-6c8b-4b1a-b36f-2689488fd4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:14.641351 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac421eb3-6c8b-4b1a-b36f-2689488fd4fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:14.642088 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac421eb3-6c8b-4b1a-b36f-2689488fd4fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:34:14.642405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac421eb3-6c8b-4b1a-b36f-2689488fd4fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:14.644020 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac421eb3-6c8b-4b1a-b36f-2689488fd4fb admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:14.657651 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 982/1968] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:34:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:34:14.662351 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632ed43-2b18-456b-9034-c4027c80cfe5 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:14.672187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632ed43-2b18-456b-9034-c4027c80cfe5 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:14.676730 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a81426-9dd3-4b40-80ed-af217f662390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:14.677390 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a81426-9dd3-4b40-80ed-af217f662390 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:14.677671 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a81426-9dd3-4b40-80ed-af217f662390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:14.677948 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a81426-9dd3-4b40-80ed-af217f662390 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:14.678641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a81426-9dd3-4b40-80ed-af217f662390 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:14.681150 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a81426-9dd3-4b40-80ed-af217f662390 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:14.710163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 987/1969] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:34:14 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:34:14.715745 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac421eb3-6c8b-4b1a-b36f-2689488fd4fb admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:14.727898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac421eb3-6c8b-4b1a-b36f-2689488fd4fb admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:14.754913 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:14.791656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59160c-873b-4a42-a73d-c71cd66459af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:14.792372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59160c-873b-4a42-a73d-c71cd66459af admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:14.792665 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59160c-873b-4a42-a73d-c71cd66459af admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:14.792963 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59160c-873b-4a42-a73d-c71cd66459af admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:14.793685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59160c-873b-4a42-a73d-c71cd66459af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:14.795661 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59160c-873b-4a42-a73d-c71cd66459af admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:14.820905 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 983/1970] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:34:14 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:34:14.823203 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a81426-9dd3-4b40-80ed-af217f662390 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:14.834439 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a81426-9dd3-4b40-80ed-af217f662390 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:14.840359 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44fc887-5951-4365-8ba8-abe7867702fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:14.841046 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fc887-5951-4365-8ba8-abe7867702fb admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:14.841329 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fc887-5951-4365-8ba8-abe7867702fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:14.841605 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fc887-5951-4365-8ba8-abe7867702fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:14.842333 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44fc887-5951-4365-8ba8-abe7867702fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:14.843936 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44fc887-5951-4365-8ba8-abe7867702fb admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:14.855457 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 988/1971] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:34:14.859684 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59160c-873b-4a42-a73d-c71cd66459af admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:14.871774 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59160c-873b-4a42-a73d-c71cd66459af admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:14.876833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2a5bb9-9c09-4303-a09a-d3c14adeb1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:14.877519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a5bb9-9c09-4303-a09a-d3c14adeb1fd admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:14.877956 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a5bb9-9c09-4303-a09a-d3c14adeb1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:14.878237 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a5bb9-9c09-4303-a09a-d3c14adeb1fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:14.878966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a5bb9-9c09-4303-a09a-d3c14adeb1fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:14.880627 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2a5bb9-9c09-4303-a09a-d3c14adeb1fd admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:14.896510 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 984/1972] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:34:14.900828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44fc887-5951-4365-8ba8-abe7867702fb admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:14.911518 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44fc887-5951-4365-8ba8-abe7867702fb admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:14.916955 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4211ecfa-8dcc-4f18-8402-24e3a5e01182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:14.917860 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211ecfa-8dcc-4f18-8402-24e3a5e01182 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:14.918142 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211ecfa-8dcc-4f18-8402-24e3a5e01182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:14.918418 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211ecfa-8dcc-4f18-8402-24e3a5e01182 admin admin] PATH_INFO: `/v3/projects/0b37b025b98d44eeb774bff2638fc48f/users/d58a8eb34c3c46f18d88d1be6e720e23/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:14.935353 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4211ecfa-8dcc-4f18-8402-24e3a5e01182 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b37b025b98d44eeb774bff2638fc48f, user_id=d58a8eb34c3c46f18d88d1be6e720e23, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:14.937989 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4211ecfa-8dcc-4f18-8402-24e3a5e01182 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:14.959294 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 989/1973] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:34:14 2022] PUT /identity/v3/projects/0b37b025b98d44eeb774bff2638fc48f/users/d58a8eb34c3c46f18d88d1be6e720e23/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:14.969763 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f26d2-ff9c-4b13-8a10-ff739a9380b9 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:14.970061 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f26d2-ff9c-4b13-8a10-ff739a9380b9 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:14.970347 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f26d2-ff9c-4b13-8a10-ff739a9380b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:14.999163 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-a87f26d2-ff9c-4b13-8a10-ff739a9380b9 None None] MFA Rules not processed for user `d58a8eb34c3c46f18d88d1be6e720e23`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:15.000455 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-a87f26d2-ff9c-4b13-8a10-ff739a9380b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:15.082761 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 985/1974] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:15.094132 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4211ecfa-8dcc-4f18-8402-24e3a5e01182 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:15.095887 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-4211ecfa-8dcc-4f18-8402-24e3a5e01182 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:15.174232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4211ecfa-8dcc-4f18-8402-24e3a5e01182 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:15.180775 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb54e3-75cc-4fe1-a24d-210ed35eb938 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:15.181674 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb54e3-75cc-4fe1-a24d-210ed35eb938 service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:15.181960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb54e3-75cc-4fe1-a24d-210ed35eb938 service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:15.182244 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb54e3-75cc-4fe1-a24d-210ed35eb938 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:15.194699 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb54e3-75cc-4fe1-a24d-210ed35eb938 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:15.197650 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb54e3-75cc-4fe1-a24d-210ed35eb938 service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:15.213523 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 990/1975] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:34:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:34:22.105904 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f26d2-ff9c-4b13-8a10-ff739a9380b9 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:22.107402 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-a87f26d2-ff9c-4b13-8a10-ff739a9380b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:22.193408 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f26d2-ff9c-4b13-8a10-ff739a9380b9 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:22.199414 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94f718-0b84-4393-9f1e-ba611b4de301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:22.201144 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94f718-0b84-4393-9f1e-ba611b4de301 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:22.201622 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94f718-0b84-4393-9f1e-ba611b4de301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:22.202077 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d94f718-0b84-4393-9f1e-ba611b4de301 admin admin] PATH_INFO: `/v3/users/94b9a4d32a034ad79852abb6a197d3d7` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:22.204851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d94f718-0b84-4393-9f1e-ba611b4de301 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94b9a4d32a034ad79852abb6a197d3d7)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:22.207737 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d94f718-0b84-4393-9f1e-ba611b4de301 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:22.276929 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:22.284502 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-9d94f718-0b84-4393-9f1e-ba611b4de301 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94b9a4d32a034ad79852abb6a197d3d7'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:22.285317 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-9d94f718-0b84-4393-9f1e-ba611b4de301 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94b9a4d32a034ad79852abb6a197d3d7'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:22.298728 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-9d94f718-0b84-4393-9f1e-ba611b4de301 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94b9a4d32a034ad79852abb6a197d3d7'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:22.313900 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-9d94f718-0b84-4393-9f1e-ba611b4de301 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94b9a4d32a034ad79852abb6a197d3d7'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:22.331670 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-9d94f718-0b84-4393-9f1e-ba611b4de301 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94b9a4d32a034ad79852abb6a197d3d7'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:22.339757 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 986/1976] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:34:22 2022] DELETE /identity/v3/users/94b9a4d32a034ad79852abb6a197d3d7 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:34:22.344371 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a709bd-63fe-4d16-ab74-e60cde7adfc8 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:22.344856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a709bd-63fe-4d16-ab74-e60cde7adfc8 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:22.344965 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a709bd-63fe-4d16-ab74-e60cde7adfc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:22.368766 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-54a709bd-63fe-4d16-ab74-e60cde7adfc8 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:22.370008 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-54a709bd-63fe-4d16-ab74-e60cde7adfc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:22.457326 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 991/1977] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:22.468248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94f718-0b84-4393-9f1e-ba611b4de301 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:22.469932 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-9d94f718-0b84-4393-9f1e-ba611b4de301 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:22.550648 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d94f718-0b84-4393-9f1e-ba611b4de301 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:22.556416 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa232cf4-016d-4442-afa3-827913d4de9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:22.557107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa232cf4-016d-4442-afa3-827913d4de9b service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:22.557394 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa232cf4-016d-4442-afa3-827913d4de9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:22.557719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa232cf4-016d-4442-afa3-827913d4de9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:22.567436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa232cf4-016d-4442-afa3-827913d4de9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:22.569141 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa232cf4-016d-4442-afa3-827913d4de9b service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:22.580875 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 987/1978] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:34:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:34:22.877456 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a709bd-63fe-4d16-ab74-e60cde7adfc8 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:22.879026 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-54a709bd-63fe-4d16-ab74-e60cde7adfc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:22.893959 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a709bd-63fe-4d16-ab74-e60cde7adfc8 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:22.899155 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba1678-a4ed-433e-8679-20936bb36ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:22.900044 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba1678-a4ed-433e-8679-20936bb36ae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:22.900352 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba1678-a4ed-433e-8679-20936bb36ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:22.900631 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba1678-a4ed-433e-8679-20936bb36ae1 admin admin] PATH_INFO: `/v3/projects/60c2bf1c616f4e0083f6f42c0d9721fb` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:22.904203 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ba1678-a4ed-433e-8679-20936bb36ae1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60c2bf1c616f4e0083f6f42c0d9721fb)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:22.904203 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ba1678-a4ed-433e-8679-20936bb36ae1 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:22.910906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:22.957465 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d1ba1678-a4ed-433e-8679-20936bb36ae1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60c2bf1c616f4e0083f6f42c0d9721fb', 'request_id': 'req-d1ba1678-a4ed-433e-8679-20936bb36ae1'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:22.964493 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d1ba1678-a4ed-433e-8679-20936bb36ae1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60c2bf1c616f4e0083f6f42c0d9721fb', 'request_id': 'req-d1ba1678-a4ed-433e-8679-20936bb36ae1'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:22.971007 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d1ba1678-a4ed-433e-8679-20936bb36ae1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60c2bf1c616f4e0083f6f42c0d9721fb', 'request_id': 'req-d1ba1678-a4ed-433e-8679-20936bb36ae1'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:22.990321 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d1ba1678-a4ed-433e-8679-20936bb36ae1 admin admin] The token cache is being invalidate because project 60c2bf1c616f4e0083f6f42c0d9721fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:34:22.990900 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d1ba1678-a4ed-433e-8679-20936bb36ae1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:22.991690 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d1ba1678-a4ed-433e-8679-20936bb36ae1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:22.993556 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 992/1979] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:34:22 2022] DELETE /identity/v3/projects/60c2bf1c616f4e0083f6f42c0d9721fb => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:34:23.000737 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a15ef-3d90-4459-8c14-8700bf9224e9 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:23.001102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a15ef-3d90-4459-8c14-8700bf9224e9 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:23.001433 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a15ef-3d90-4459-8c14-8700bf9224e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:23.019759 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-4c6a15ef-3d90-4459-8c14-8700bf9224e9 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:23.024234 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-4c6a15ef-3d90-4459-8c14-8700bf9224e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:23.093826 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 988/1980] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:23.103129 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe647e5-3b76-42a9-a6a6-1caeef637ff8 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:23.103426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe647e5-3b76-42a9-a6a6-1caeef637ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:23.103713 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe647e5-3b76-42a9-a6a6-1caeef637ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:23.125769 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-4fe647e5-3b76-42a9-a6a6-1caeef637ff8 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:23.126981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-4fe647e5-3b76-42a9-a6a6-1caeef637ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:23.135926 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 993/1981] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:34:23.140662 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a15ef-3d90-4459-8c14-8700bf9224e9 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:23.153393 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a15ef-3d90-4459-8c14-8700bf9224e9 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:23.161170 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03167b-0790-47e8-a298-85d618f5f58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:23.161850 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03167b-0790-47e8-a298-85d618f5f58a admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:23.162140 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03167b-0790-47e8-a298-85d618f5f58a admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:23.162423 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03167b-0790-47e8-a298-85d618f5f58a admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:23.163165 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b03167b-0790-47e8-a298-85d618f5f58a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:34:23.163474 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b03167b-0790-47e8-a298-85d618f5f58a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:23.165537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b03167b-0790-47e8-a298-85d618f5f58a admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:23.179442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 989/1982] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:34:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:34:23.184235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe647e5-3b76-42a9-a6a6-1caeef637ff8 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:23.194155 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe647e5-3b76-42a9-a6a6-1caeef637ff8 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:23.199964 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc98ea5-ee78-4406-9bcb-007193efba51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:23.201156 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc98ea5-ee78-4406-9bcb-007193efba51 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:23.201438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc98ea5-ee78-4406-9bcb-007193efba51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:23.201712 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc98ea5-ee78-4406-9bcb-007193efba51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:23.202559 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc98ea5-ee78-4406-9bcb-007193efba51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:23.204543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc98ea5-ee78-4406-9bcb-007193efba51 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:23.234351 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 994/1983] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:34:23 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:34:23.239305 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03167b-0790-47e8-a298-85d618f5f58a admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:23.249805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03167b-0790-47e8-a298-85d618f5f58a admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:23.297610 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:23.313728 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37030b27-4675-4552-acf1-257f20b0c930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:23.314408 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37030b27-4675-4552-acf1-257f20b0c930 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:23.314699 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37030b27-4675-4552-acf1-257f20b0c930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:23.314980 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37030b27-4675-4552-acf1-257f20b0c930 admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:23.315671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37030b27-4675-4552-acf1-257f20b0c930 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:23.317686 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37030b27-4675-4552-acf1-257f20b0c930 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:23.342692 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 990/1984] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:34:23 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:34:23.346276 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc98ea5-ee78-4406-9bcb-007193efba51 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:23.356824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc98ea5-ee78-4406-9bcb-007193efba51 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:23.361682 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27435c54-8f4f-4e49-924d-5b7bbaf00379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:23.362352 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27435c54-8f4f-4e49-924d-5b7bbaf00379 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:23.362639 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27435c54-8f4f-4e49-924d-5b7bbaf00379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:23.362920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27435c54-8f4f-4e49-924d-5b7bbaf00379 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:23.363645 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27435c54-8f4f-4e49-924d-5b7bbaf00379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:23.365324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27435c54-8f4f-4e49-924d-5b7bbaf00379 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:23.377066 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 995/1985] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:34:23.381255 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37030b27-4675-4552-acf1-257f20b0c930 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:23.393117 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37030b27-4675-4552-acf1-257f20b0c930 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:23.398001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94cc4e0-dcca-447a-a7ef-244452677dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:23.399263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cc4e0-dcca-447a-a7ef-244452677dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:23.399684 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cc4e0-dcca-447a-a7ef-244452677dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:23.399981 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cc4e0-dcca-447a-a7ef-244452677dc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:23.400753 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94cc4e0-dcca-447a-a7ef-244452677dc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:23.402360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94cc4e0-dcca-447a-a7ef-244452677dc9 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:23.413071 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 991/1986] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:34:23.417404 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27435c54-8f4f-4e49-924d-5b7bbaf00379 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:23.427530 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27435c54-8f4f-4e49-924d-5b7bbaf00379 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:23.434332 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8923aee-dd0e-48ef-98d6-310020623674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:23.435237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8923aee-dd0e-48ef-98d6-310020623674 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:23.435541 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8923aee-dd0e-48ef-98d6-310020623674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:23.435799 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8923aee-dd0e-48ef-98d6-310020623674 admin admin] PATH_INFO: `/v3/projects/12381a74e9c642b7ac4e5a77bd8ea850/users/e05f99049bd54038a92e1a3776df66f6/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:23.453324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8923aee-dd0e-48ef-98d6-310020623674 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12381a74e9c642b7ac4e5a77bd8ea850, user_id=e05f99049bd54038a92e1a3776df66f6, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:23.455972 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8923aee-dd0e-48ef-98d6-310020623674 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:23.470496 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 996/1987] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:34:23 2022] PUT /identity/v3/projects/12381a74e9c642b7ac4e5a77bd8ea850/users/e05f99049bd54038a92e1a3776df66f6/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:34:23.481394 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c3c22-2e7c-4460-9f50-119f775ae38d None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:23.481717 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c3c22-2e7c-4460-9f50-119f775ae38d None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:23.482049 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c3c22-2e7c-4460-9f50-119f775ae38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:23.516940 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-fc4c3c22-2e7c-4460-9f50-119f775ae38d None None] MFA Rules not processed for user `e05f99049bd54038a92e1a3776df66f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:23.518299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-fc4c3c22-2e7c-4460-9f50-119f775ae38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:23.599959 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 992/1988] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:23.611410 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8923aee-dd0e-48ef-98d6-310020623674 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:23.613799 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a8923aee-dd0e-48ef-98d6-310020623674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:23.690458 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8923aee-dd0e-48ef-98d6-310020623674 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:23.694256 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5512dcd-7fd3-459d-8835-a6239cf78b82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:23.697263 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5512dcd-7fd3-459d-8835-a6239cf78b82 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:23.697263 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5512dcd-7fd3-459d-8835-a6239cf78b82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:23.697263 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5512dcd-7fd3-459d-8835-a6239cf78b82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:23.706244 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5512dcd-7fd3-459d-8835-a6239cf78b82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:23.707676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5512dcd-7fd3-459d-8835-a6239cf78b82 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:23.718901 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 997/1989] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:34:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:34:23.751355 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c3c22-2e7c-4460-9f50-119f775ae38d None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:23.835511 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c3c22-2e7c-4460-9f50-119f775ae38d None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:23.849652 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ed935-4090-4bd1-a60e-66b7e8e4660f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:23.850743 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ed935-4090-4bd1-a60e-66b7e8e4660f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:23.851229 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ed935-4090-4bd1-a60e-66b7e8e4660f admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:23.852271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ed935-4090-4bd1-a60e-66b7e8e4660f admin admin] PATH_INFO: `/v3/users/e05f99049bd54038a92e1a3776df66f6` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:23.853613 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497ed935-4090-4bd1-a60e-66b7e8e4660f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e05f99049bd54038a92e1a3776df66f6)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:23.855523 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497ed935-4090-4bd1-a60e-66b7e8e4660f admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:23.911910 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-497ed935-4090-4bd1-a60e-66b7e8e4660f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e05f99049bd54038a92e1a3776df66f6'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:23.912757 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-497ed935-4090-4bd1-a60e-66b7e8e4660f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e05f99049bd54038a92e1a3776df66f6'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:23.922973 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-497ed935-4090-4bd1-a60e-66b7e8e4660f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e05f99049bd54038a92e1a3776df66f6'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:23.924143 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:23.941295 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-497ed935-4090-4bd1-a60e-66b7e8e4660f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e05f99049bd54038a92e1a3776df66f6'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:23.954133 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-497ed935-4090-4bd1-a60e-66b7e8e4660f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e05f99049bd54038a92e1a3776df66f6'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:23.963496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 993/1990] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:34:23 2022] DELETE /identity/v3/users/e05f99049bd54038a92e1a3776df66f6 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:23.968734 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8012b47-5d23-4fcb-8e3e-6e8593bbbb43 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:23.969269 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8012b47-5d23-4fcb-8e3e-6e8593bbbb43 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:23.969786 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8012b47-5d23-4fcb-8e3e-6e8593bbbb43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:23.996079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a8012b47-5d23-4fcb-8e3e-6e8593bbbb43 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:23.996079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a8012b47-5d23-4fcb-8e3e-6e8593bbbb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:24.037639 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:24.081952 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 998/1991] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:24.093118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ed935-4090-4bd1-a60e-66b7e8e4660f admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:24.095216 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-497ed935-4090-4bd1-a60e-66b7e8e4660f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:24.167639 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ed935-4090-4bd1-a60e-66b7e8e4660f admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:24.171318 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea46cac-f8f7-4114-a335-b2279e88778a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:24.171992 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea46cac-f8f7-4114-a335-b2279e88778a service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:24.172307 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea46cac-f8f7-4114-a335-b2279e88778a service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:24.172594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea46cac-f8f7-4114-a335-b2279e88778a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:24.201913 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea46cac-f8f7-4114-a335-b2279e88778a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:24.205201 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea46cac-f8f7-4114-a335-b2279e88778a service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:24.219399 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 994/1992] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:34:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:34:24.587450 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8012b47-5d23-4fcb-8e3e-6e8593bbbb43 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:24.590046 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a8012b47-5d23-4fcb-8e3e-6e8593bbbb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:24.603264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8012b47-5d23-4fcb-8e3e-6e8593bbbb43 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:24.608399 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15df37f-4ca4-4fe5-86d1-b501d068f890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:24.609288 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15df37f-4ca4-4fe5-86d1-b501d068f890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:24.609813 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15df37f-4ca4-4fe5-86d1-b501d068f890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:24.610120 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15df37f-4ca4-4fe5-86d1-b501d068f890 admin admin] PATH_INFO: `/v3/projects/12381a74e9c642b7ac4e5a77bd8ea850` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:24.611362 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15df37f-4ca4-4fe5-86d1-b501d068f890 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12381a74e9c642b7ac4e5a77bd8ea850)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:24.614064 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15df37f-4ca4-4fe5-86d1-b501d068f890 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:24.670548 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e15df37f-4ca4-4fe5-86d1-b501d068f890 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12381a74e9c642b7ac4e5a77bd8ea850', 'request_id': 'req-e15df37f-4ca4-4fe5-86d1-b501d068f890'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:24.676370 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e15df37f-4ca4-4fe5-86d1-b501d068f890 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12381a74e9c642b7ac4e5a77bd8ea850', 'request_id': 'req-e15df37f-4ca4-4fe5-86d1-b501d068f890'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:24.684642 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e15df37f-4ca4-4fe5-86d1-b501d068f890 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12381a74e9c642b7ac4e5a77bd8ea850', 'request_id': 'req-e15df37f-4ca4-4fe5-86d1-b501d068f890'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:24.699909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e15df37f-4ca4-4fe5-86d1-b501d068f890 admin admin] The token cache is being invalidate because project 12381a74e9c642b7ac4e5a77bd8ea850 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:34:24.700213 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e15df37f-4ca4-4fe5-86d1-b501d068f890 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:24.701169 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e15df37f-4ca4-4fe5-86d1-b501d068f890 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:24.702731 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 999/1993] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:34:24 2022] DELETE /identity/v3/projects/12381a74e9c642b7ac4e5a77bd8ea850 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:24.707877 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f049246-716b-4162-92cc-6fa4504b9ceb None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:24.708211 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f049246-716b-4162-92cc-6fa4504b9ceb None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:24.708513 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f049246-716b-4162-92cc-6fa4504b9ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:24.728424 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-3f049246-716b-4162-92cc-6fa4504b9ceb None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:24.729640 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-3f049246-716b-4162-92cc-6fa4504b9ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:24.835508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 995/1994] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:34:24.846506 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4895f93-a919-4326-87dc-04e7a931cf6e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:24.846506 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4895f93-a919-4326-87dc-04e7a931cf6e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:24.846506 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4895f93-a919-4326-87dc-04e7a931cf6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:24.851218 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f049246-716b-4162-92cc-6fa4504b9ceb None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:24.853984 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-3f049246-716b-4162-92cc-6fa4504b9ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:24.867804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-b4895f93-a919-4326-87dc-04e7a931cf6e None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:24.869855 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b4895f93-a919-4326-87dc-04e7a931cf6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:24.871302 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f049246-716b-4162-92cc-6fa4504b9ceb None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:24.878669 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0682d63-42e3-44df-ac19-903b1958acbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:24.880159 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1000/1995] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:34:24.880484 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0682d63-42e3-44df-ac19-903b1958acbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:24.880484 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0682d63-42e3-44df-ac19-903b1958acbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:24.881452 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0682d63-42e3-44df-ac19-903b1958acbc admin admin] PATH_INFO: `/v3/users/d58a8eb34c3c46f18d88d1be6e720e23` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:24.882408 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0682d63-42e3-44df-ac19-903b1958acbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d58a8eb34c3c46f18d88d1be6e720e23)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:24.884976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4895f93-a919-4326-87dc-04e7a931cf6e None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:24.885771 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0682d63-42e3-44df-ac19-903b1958acbc admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:24.897877 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4895f93-a919-4326-87dc-04e7a931cf6e None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:24.902740 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd56173-5bb4-40d5-8a07-2d98021afaf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:24.903412 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd56173-5bb4-40d5-8a07-2d98021afaf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:24.903695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd56173-5bb4-40d5-8a07-2d98021afaf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:24.903986 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd56173-5bb4-40d5-8a07-2d98021afaf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:24.904777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd56173-5bb4-40d5-8a07-2d98021afaf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:34:24.905087 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd56173-5bb4-40d5-8a07-2d98021afaf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:24.906695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd56173-5bb4-40d5-8a07-2d98021afaf6 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:24.920154 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1001/1996] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:34:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:34:24.924557 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd56173-5bb4-40d5-8a07-2d98021afaf6 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:24.935232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd56173-5bb4-40d5-8a07-2d98021afaf6 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:24.939878 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0202f9f-ea73-411f-bc12-c09eca98ec97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:24.940568 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0202f9f-ea73-411f-bc12-c09eca98ec97 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:24.940950 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0202f9f-ea73-411f-bc12-c09eca98ec97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:24.941166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0202f9f-ea73-411f-bc12-c09eca98ec97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:24.941866 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0202f9f-ea73-411f-bc12-c09eca98ec97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:24.943766 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0202f9f-ea73-411f-bc12-c09eca98ec97 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:24.947358 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-c0682d63-42e3-44df-ac19-903b1958acbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd58a8eb34c3c46f18d88d1be6e720e23'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:24.948098 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-c0682d63-42e3-44df-ac19-903b1958acbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd58a8eb34c3c46f18d88d1be6e720e23'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:24.955668 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-c0682d63-42e3-44df-ac19-903b1958acbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd58a8eb34c3c46f18d88d1be6e720e23'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:24.969746 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:24.971095 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-c0682d63-42e3-44df-ac19-903b1958acbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd58a8eb34c3c46f18d88d1be6e720e23'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:24.975596 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1002/1997] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:34:24 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:34:24.977331 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0202f9f-ea73-411f-bc12-c09eca98ec97 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:24.983455 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-c0682d63-42e3-44df-ac19-903b1958acbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd58a8eb34c3c46f18d88d1be6e720e23'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:24.990077 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 996/1998] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:34:24 2022] DELETE /identity/v3/users/d58a8eb34c3c46f18d88d1be6e720e23 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:24.994875 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad0748-9f62-42c2-b0db-687b60a1b408 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:24.995154 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad0748-9f62-42c2-b0db-687b60a1b408 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:24.996802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad0748-9f62-42c2-b0db-687b60a1b408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:25.013553 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-9bad0748-9f62-42c2-b0db-687b60a1b408 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:25.014770 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-9bad0748-9f62-42c2-b0db-687b60a1b408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:25.052021 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0202f9f-ea73-411f-bc12-c09eca98ec97 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:25.071763 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:25.089642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 997/1999] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:34:25.099545 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad0748-9f62-42c2-b0db-687b60a1b408 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:25.101011 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-9bad0748-9f62-42c2-b0db-687b60a1b408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:25.116290 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb62c84-1e45-4059-ac89-f3c0e2bcbb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:25.117023 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb62c84-1e45-4059-ac89-f3c0e2bcbb59 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:25.117451 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb62c84-1e45-4059-ac89-f3c0e2bcbb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:25.117936 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb62c84-1e45-4059-ac89-f3c0e2bcbb59 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:25.118814 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb62c84-1e45-4059-ac89-f3c0e2bcbb59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:25.120945 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb62c84-1e45-4059-ac89-f3c0e2bcbb59 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:25.143848 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1003/2000] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:34:24 2022] POST /identity/v3/users => generated 288 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:34:25.148090 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb62c84-1e45-4059-ac89-f3c0e2bcbb59 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:25.152777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-bdb62c84-1e45-4059-ac89-f3c0e2bcbb59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:25.168309 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad0748-9f62-42c2-b0db-687b60a1b408 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:25.168874 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb62c84-1e45-4059-ac89-f3c0e2bcbb59 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:25.172002 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d12523-d2fe-48aa-8920-89e8bb236785 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:25.172687 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d12523-d2fe-48aa-8920-89e8bb236785 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:25.172976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d12523-d2fe-48aa-8920-89e8bb236785 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:25.173256 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d12523-d2fe-48aa-8920-89e8bb236785 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:25.173389 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60665743-ea44-4577-bead-bc9b56d597a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:25.174056 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60665743-ea44-4577-bead-bc9b56d597a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:25.174343 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60665743-ea44-4577-bead-bc9b56d597a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:25.174672 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60665743-ea44-4577-bead-bc9b56d597a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:25.175336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60665743-ea44-4577-bead-bc9b56d597a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:25.176985 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60665743-ea44-4577-bead-bc9b56d597a5 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:25.182916 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d12523-d2fe-48aa-8920-89e8bb236785 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:25.184509 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d12523-d2fe-48aa-8920-89e8bb236785 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:25.188285 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1004/2001] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:34:25.192262 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60665743-ea44-4577-bead-bc9b56d597a5 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:25.195081 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 998/2002] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:34:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:34:25.203219 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60665743-ea44-4577-bead-bc9b56d597a5 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:25.208230 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e3bdf2-d92d-4033-bae1-cf38dae2defd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:25.208964 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3bdf2-d92d-4033-bae1-cf38dae2defd admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:25.209250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3bdf2-d92d-4033-bae1-cf38dae2defd admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:25.209536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e3bdf2-d92d-4033-bae1-cf38dae2defd admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:25.210288 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e3bdf2-d92d-4033-bae1-cf38dae2defd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:25.212493 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e3bdf2-d92d-4033-bae1-cf38dae2defd admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:25.226361 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1005/2003] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:34:25.227499 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d12523-d2fe-48aa-8920-89e8bb236785 service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:25.237267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d12523-d2fe-48aa-8920-89e8bb236785 service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:25.244302 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca96da9-4eb5-4a2a-b7ed-91b13eea0909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:25.245221 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca96da9-4eb5-4a2a-b7ed-91b13eea0909 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:25.245512 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca96da9-4eb5-4a2a-b7ed-91b13eea0909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:25.245798 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca96da9-4eb5-4a2a-b7ed-91b13eea0909 admin admin] PATH_INFO: `/v3/projects/61516a3c24f543d68bdb3724a11f7042/users/c819599f00844589b2957d9587c62d11/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:25.267152 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca96da9-4eb5-4a2a-b7ed-91b13eea0909 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61516a3c24f543d68bdb3724a11f7042, user_id=c819599f00844589b2957d9587c62d11, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:25.269874 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca96da9-4eb5-4a2a-b7ed-91b13eea0909 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:25.293722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 999/2004] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:34:25 2022] PUT /identity/v3/projects/61516a3c24f543d68bdb3724a11f7042/users/c819599f00844589b2957d9587c62d11/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:25.306879 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e07c1-1bfe-4dd9-8caa-9c1988e167f8 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:25.307414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e07c1-1bfe-4dd9-8caa-9c1988e167f8 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:25.307975 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825e07c1-1bfe-4dd9-8caa-9c1988e167f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:25.352291 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-825e07c1-1bfe-4dd9-8caa-9c1988e167f8 None None] MFA Rules not processed for user `c819599f00844589b2957d9587c62d11`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:25.356429 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-825e07c1-1bfe-4dd9-8caa-9c1988e167f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:25.453117 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1006/2005] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:25.467203 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca96da9-4eb5-4a2a-b7ed-91b13eea0909 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:25.469072 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-3ca96da9-4eb5-4a2a-b7ed-91b13eea0909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:25.537736 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e07c1-1bfe-4dd9-8caa-9c1988e167f8 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:25.541154 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-825e07c1-1bfe-4dd9-8caa-9c1988e167f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:25.557469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca96da9-4eb5-4a2a-b7ed-91b13eea0909 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:25.562740 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a928fe3-deb6-4094-af63-4da94e90dbd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:25.563646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a928fe3-deb6-4094-af63-4da94e90dbd0 service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:25.564123 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a928fe3-deb6-4094-af63-4da94e90dbd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:25.564647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a928fe3-deb6-4094-af63-4da94e90dbd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:25.592066 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a928fe3-deb6-4094-af63-4da94e90dbd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:25.594168 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a928fe3-deb6-4094-af63-4da94e90dbd0 service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:25.620021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1000/2006] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:34:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:34:25.652036 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825e07c1-1bfe-4dd9-8caa-9c1988e167f8 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:25.659736 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6e3f4-bd22-4522-9836-feebbc125e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:25.660762 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6e3f4-bd22-4522-9836-feebbc125e07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:25.661071 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6e3f4-bd22-4522-9836-feebbc125e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:25.661317 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6e3f4-bd22-4522-9836-feebbc125e07 admin admin] PATH_INFO: `/v3/projects/0b37b025b98d44eeb774bff2638fc48f` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:25.662854 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6e3f4-bd22-4522-9836-feebbc125e07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b37b025b98d44eeb774bff2638fc48f)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:25.665462 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6e3f4-bd22-4522-9836-feebbc125e07 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:25.697849 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:25.736493 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-bda6e3f4-bd22-4522-9836-feebbc125e07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b37b025b98d44eeb774bff2638fc48f', 'request_id': 'req-bda6e3f4-bd22-4522-9836-feebbc125e07'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:25.750765 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-bda6e3f4-bd22-4522-9836-feebbc125e07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b37b025b98d44eeb774bff2638fc48f', 'request_id': 'req-bda6e3f4-bd22-4522-9836-feebbc125e07'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:25.760316 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-bda6e3f4-bd22-4522-9836-feebbc125e07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b37b025b98d44eeb774bff2638fc48f', 'request_id': 'req-bda6e3f4-bd22-4522-9836-feebbc125e07'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:25.788892 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-bda6e3f4-bd22-4522-9836-feebbc125e07 admin admin] The token cache is being invalidate because project 0b37b025b98d44eeb774bff2638fc48f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:34:25.789529 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-bda6e3f4-bd22-4522-9836-feebbc125e07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:25.790460 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-bda6e3f4-bd22-4522-9836-feebbc125e07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:25.793249 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1007/2007] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:34:25 2022] DELETE /identity/v3/projects/0b37b025b98d44eeb774bff2638fc48f => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:25.798465 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f31fe-0d1e-4659-a005-35ead318a7c6 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:25.798869 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f31fe-0d1e-4659-a005-35ead318a7c6 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:25.799144 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f31fe-0d1e-4659-a005-35ead318a7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:25.829608 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-5a4f31fe-0d1e-4659-a005-35ead318a7c6 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:25.830952 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-5a4f31fe-0d1e-4659-a005-35ead318a7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:25.840707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:25.995995 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1001/2008] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:26.004985 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc97ae-8089-437b-82fd-1d88a73eab79 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:26.005284 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc97ae-8089-437b-82fd-1d88a73eab79 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:26.005577 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc97ae-8089-437b-82fd-1d88a73eab79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:26.031768 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-53dc97ae-8089-437b-82fd-1d88a73eab79 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:26.033235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-53dc97ae-8089-437b-82fd-1d88a73eab79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:26.049950 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1008/2009] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:26.056301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f31fe-0d1e-4659-a005-35ead318a7c6 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:26.070806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f31fe-0d1e-4659-a005-35ead318a7c6 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:26.077227 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6dcf59-b6c1-4804-a87a-6d3542eb40fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:26.077920 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6dcf59-b6c1-4804-a87a-6d3542eb40fc admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:26.078792 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6dcf59-b6c1-4804-a87a-6d3542eb40fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:26.079155 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6dcf59-b6c1-4804-a87a-6d3542eb40fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:26.079913 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6dcf59-b6c1-4804-a87a-6d3542eb40fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:34:26.080258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6dcf59-b6c1-4804-a87a-6d3542eb40fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:26.082032 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6dcf59-b6c1-4804-a87a-6d3542eb40fc admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:26.111493 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1002/2010] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:34:26.113559 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc97ae-8089-437b-82fd-1d88a73eab79 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:26.128317 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc97ae-8089-437b-82fd-1d88a73eab79 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:26.136041 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ffb1b1-543c-46ee-bbc7-d9358da3d820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:26.136757 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffb1b1-543c-46ee-bbc7-d9358da3d820 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:26.137042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffb1b1-543c-46ee-bbc7-d9358da3d820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:26.137356 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffb1b1-543c-46ee-bbc7-d9358da3d820 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:26.138044 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ffb1b1-543c-46ee-bbc7-d9358da3d820 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:26.141813 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ffb1b1-543c-46ee-bbc7-d9358da3d820 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:26.185974 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1009/2011] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:34:26 2022] POST /identity/v3/projects => generated 373 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:34:26.190428 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6dcf59-b6c1-4804-a87a-6d3542eb40fc admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:26.202604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6dcf59-b6c1-4804-a87a-6d3542eb40fc admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:26.281540 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ac5a2-256b-4c4d-bedc-392a1b392b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:26.282268 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ac5a2-256b-4c4d-bedc-392a1b392b0a admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:26.282557 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ac5a2-256b-4c4d-bedc-392a1b392b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:26.282966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ac5a2-256b-4c4d-bedc-392a1b392b0a admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:26.283689 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ac5a2-256b-4c4d-bedc-392a1b392b0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:26.286368 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ac5a2-256b-4c4d-bedc-392a1b392b0a admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:26.319004 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1003/2012] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:34:26 2022] POST /identity/v3/users => generated 287 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:34:26.324026 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ffb1b1-543c-46ee-bbc7-d9358da3d820 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:26.339779 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ffb1b1-543c-46ee-bbc7-d9358da3d820 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:26.346172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed947429-4bd4-4306-92f8-d08f073757f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:26.346970 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed947429-4bd4-4306-92f8-d08f073757f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:26.347285 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed947429-4bd4-4306-92f8-d08f073757f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:26.347676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed947429-4bd4-4306-92f8-d08f073757f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:26.348824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed947429-4bd4-4306-92f8-d08f073757f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:26.350597 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed947429-4bd4-4306-92f8-d08f073757f1 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:26.365352 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1010/2013] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:34:26.370744 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ac5a2-256b-4c4d-bedc-392a1b392b0a admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:26.385646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ac5a2-256b-4c4d-bedc-392a1b392b0a admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:26.393807 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebeda4-8f12-43a9-ae43-b5db11f0f882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:26.394994 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebeda4-8f12-43a9-ae43-b5db11f0f882 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:26.395516 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebeda4-8f12-43a9-ae43-b5db11f0f882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:26.396045 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebeda4-8f12-43a9-ae43-b5db11f0f882 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:26.397547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ebeda4-8f12-43a9-ae43-b5db11f0f882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:26.400596 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ebeda4-8f12-43a9-ae43-b5db11f0f882 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:26.413468 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1004/2014] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:34:26.418486 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed947429-4bd4-4306-92f8-d08f073757f1 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:26.429304 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed947429-4bd4-4306-92f8-d08f073757f1 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:26.434888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d12699-6b36-4d73-a159-066a97316018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:26.435792 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d12699-6b36-4d73-a159-066a97316018 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:26.436078 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d12699-6b36-4d73-a159-066a97316018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:26.436390 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d12699-6b36-4d73-a159-066a97316018 admin admin] PATH_INFO: `/v3/projects/c5c98a8a099f4628acf38de147718198/users/e981c1fed59d43c48981ad7ddcafb234/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:26.456339 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d12699-6b36-4d73-a159-066a97316018 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5c98a8a099f4628acf38de147718198, user_id=e981c1fed59d43c48981ad7ddcafb234, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:26.462336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d12699-6b36-4d73-a159-066a97316018 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:26.480735 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1011/2015] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:34:26 2022] PUT /identity/v3/projects/c5c98a8a099f4628acf38de147718198/users/e981c1fed59d43c48981ad7ddcafb234/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:26.494589 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030f8c5-50de-4e84-a674-4bcebf00afed None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:26.494917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030f8c5-50de-4e84-a674-4bcebf00afed None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:26.495201 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030f8c5-50de-4e84-a674-4bcebf00afed None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:26.498563 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1005/2016] 10.222.0.16 () {54 vars in 921 bytes} [Mon May 23 12:34:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:34:26.500454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6767b432-d02a-49de-ac26-d7e7ca0ec64d None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:26.500800 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6767b432-d02a-49de-ac26-d7e7ca0ec64d None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:26.501106 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6767b432-d02a-49de-ac26-d7e7ca0ec64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:26.536767 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f16af-03ea-4b44-8dc2-c996bb1cb7c3 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:26.537218 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f16af-03ea-4b44-8dc2-c996bb1cb7c3 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:26.537606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f16af-03ea-4b44-8dc2-c996bb1cb7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:26.540161 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-6767b432-d02a-49de-ac26-d7e7ca0ec64d None None] MFA Rules not processed for user `e981c1fed59d43c48981ad7ddcafb234`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:26.541436 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-6767b432-d02a-49de-ac26-d7e7ca0ec64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:26.564044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-3c0f16af-03ea-4b44-8dc2-c996bb1cb7c3 None None] MFA Rules not processed for user `ebc0f137e0d24dd2a460689bc17a1b98`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:26.565391 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-3c0f16af-03ea-4b44-8dc2-c996bb1cb7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:26.632348 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1012/2017] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:26.638108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6767b432-d02a-49de-ac26-d7e7ca0ec64d None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:26.667677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1006/2018] 10.222.0.16 () {58 vars in 1018 bytes} [Mon May 23 12:34:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:26.680250 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f16af-03ea-4b44-8dc2-c996bb1cb7c3 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:26.682163 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-3c0f16af-03ea-4b44-8dc2-c996bb1cb7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:26.692134 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:26.696545 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f16af-03ea-4b44-8dc2-c996bb1cb7c3 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:26.702725 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f28ecc-9f60-4d0f-a823-a54efe5bd402 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc0f137e0d24dd2a460689bc17a1b98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:26.703404 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f28ecc-9f60-4d0f-a823-a54efe5bd402 service placement] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:26.703691 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f28ecc-9f60-4d0f-a823-a54efe5bd402 service placement] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:26.703973 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f28ecc-9f60-4d0f-a823-a54efe5bd402 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:26.715968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f28ecc-9f60-4d0f-a823-a54efe5bd402 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:26.717538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f28ecc-9f60-4d0f-a823-a54efe5bd402 service placement] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:26.723486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:26.726204 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6767b432-d02a-49de-ac26-d7e7ca0ec64d None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:26.731614 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1007/2019] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:34:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:34:26.733658 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a39cc06-9f70-439f-8a33-6b07a4c1a5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:26.734342 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a39cc06-9f70-439f-8a33-6b07a4c1a5b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:26.734629 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a39cc06-9f70-439f-8a33-6b07a4c1a5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:26.734908 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a39cc06-9f70-439f-8a33-6b07a4c1a5b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:26.735612 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a39cc06-9f70-439f-8a33-6b07a4c1a5b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:26.737621 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a39cc06-9f70-439f-8a33-6b07a4c1a5b9 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:26.774639 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1013/2020] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:34:26 2022] POST /identity/v3/projects => generated 371 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:34:26.779924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f28ecc-9f60-4d0f-a823-a54efe5bd402 service placement] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:26.794177 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f28ecc-9f60-4d0f-a823-a54efe5bd402 service placement] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:26.878906 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64576af6-7d74-4efb-b059-7aa7640bfd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:26.879596 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64576af6-7d74-4efb-b059-7aa7640bfd74 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:26.879886 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64576af6-7d74-4efb-b059-7aa7640bfd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:26.880187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64576af6-7d74-4efb-b059-7aa7640bfd74 admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:26.880896 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64576af6-7d74-4efb-b059-7aa7640bfd74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:26.883050 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64576af6-7d74-4efb-b059-7aa7640bfd74 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:26.910847 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1008/2021] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:34:26 2022] POST /identity/v3/users => generated 292 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:34:26.912730 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a39cc06-9f70-439f-8a33-6b07a4c1a5b9 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:26.925849 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a39cc06-9f70-439f-8a33-6b07a4c1a5b9 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:26.932590 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db75196-dfad-4fef-99fe-c1173d442991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:26.933977 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db75196-dfad-4fef-99fe-c1173d442991 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:26.934269 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db75196-dfad-4fef-99fe-c1173d442991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:26.934611 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db75196-dfad-4fef-99fe-c1173d442991 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:26.935351 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db75196-dfad-4fef-99fe-c1173d442991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:26.937004 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db75196-dfad-4fef-99fe-c1173d442991 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:26.953227 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1014/2022] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:34:26.954410 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64576af6-7d74-4efb-b059-7aa7640bfd74 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:26.965710 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64576af6-7d74-4efb-b059-7aa7640bfd74 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:26.971005 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f84787-83ef-4634-9ba0-f79c9e8f0037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:26.971833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f84787-83ef-4634-9ba0-f79c9e8f0037 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:26.972119 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f84787-83ef-4634-9ba0-f79c9e8f0037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:26.972430 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f84787-83ef-4634-9ba0-f79c9e8f0037 admin admin] PATH_INFO: `/v3/projects/2f9687dbe3b34f259c044297eaf3ce04/users/cc557049aadf44b6b250c8e08ed77d31/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:26.996566 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f84787-83ef-4634-9ba0-f79c9e8f0037 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f9687dbe3b34f259c044297eaf3ce04, user_id=cc557049aadf44b6b250c8e08ed77d31, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:26.998951 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f84787-83ef-4634-9ba0-f79c9e8f0037 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:27.019079 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1009/2023] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:34:26 2022] PUT /identity/v3/projects/2f9687dbe3b34f259c044297eaf3ce04/users/cc557049aadf44b6b250c8e08ed77d31/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:27.025395 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db75196-dfad-4fef-99fe-c1173d442991 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:27.041880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db75196-dfad-4fef-99fe-c1173d442991 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:27.121370 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a520e-f495-414a-aded-f0523b4e9a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:27.122172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a520e-f495-414a-aded-f0523b4e9a68 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:27.122525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a520e-f495-414a-aded-f0523b4e9a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:27.123551 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a520e-f495-414a-aded-f0523b4e9a68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:27.124444 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a520e-f495-414a-aded-f0523b4e9a68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:27.126103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a520e-f495-414a-aded-f0523b4e9a68 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:27.140595 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1015/2024] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:34:27.142092 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f84787-83ef-4634-9ba0-f79c9e8f0037 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:27.162214 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f84787-83ef-4634-9ba0-f79c9e8f0037 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:27.167576 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6cf17-193c-4735-b4f1-887aa5b1af4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:27.169040 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6cf17-193c-4735-b4f1-887aa5b1af4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:27.169324 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6cf17-193c-4735-b4f1-887aa5b1af4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:27.169605 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6cf17-193c-4735-b4f1-887aa5b1af4d admin admin] PATH_INFO: `/v3/projects/2f9687dbe3b34f259c044297eaf3ce04/users/cc557049aadf44b6b250c8e08ed77d31/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:27.171689 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a6cf17-193c-4735-b4f1-887aa5b1af4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f9687dbe3b34f259c044297eaf3ce04, user_id=cc557049aadf44b6b250c8e08ed77d31, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:27.174486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a6cf17-193c-4735-b4f1-887aa5b1af4d admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:27.186950 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1010/2025] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:34:27 2022] PUT /identity/v3/projects/2f9687dbe3b34f259c044297eaf3ce04/users/cc557049aadf44b6b250c8e08ed77d31/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:34:27.200165 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5ab90-cc7e-49c6-8045-126b079910ab None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:27.200738 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5ab90-cc7e-49c6-8045-126b079910ab None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:27.201025 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5ab90-cc7e-49c6-8045-126b079910ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:27.218209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b5b25-3753-41cc-a9d5-c009b1e85e9a None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:27.218741 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b5b25-3753-41cc-a9d5-c009b1e85e9a None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:27.218741 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b5b25-3753-41cc-a9d5-c009b1e85e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:27.241548 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-7ac5ab90-cc7e-49c6-8045-126b079910ab None None] MFA Rules not processed for user `cc557049aadf44b6b250c8e08ed77d31`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:27.244206 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-7ac5ab90-cc7e-49c6-8045-126b079910ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:27.246625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-076b5b25-3753-41cc-a9d5-c009b1e85e9a None None] MFA Rules not processed for user `c819599f00844589b2957d9587c62d11`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:27.247705 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-076b5b25-3753-41cc-a9d5-c009b1e85e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:27.321791 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1011/2026] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:34:27.322918 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076b5b25-3753-41cc-a9d5-c009b1e85e9a None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:27.324295 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-076b5b25-3753-41cc-a9d5-c009b1e85e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:27.351175 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1016/2027] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:27 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:27.363361 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac5ab90-cc7e-49c6-8045-126b079910ab None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:27.365218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-7ac5ab90-cc7e-49c6-8045-126b079910ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:27.407253 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:27.419501 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:27.425040 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076b5b25-3753-41cc-a9d5-c009b1e85e9a None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:27.440697 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9c62e-0c45-4752-9516-0b34ae1d9b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:27.442063 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9c62e-0c45-4752-9516-0b34ae1d9b27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:27.442580 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9c62e-0c45-4752-9516-0b34ae1d9b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:27.442719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac5ab90-cc7e-49c6-8045-126b079910ab None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:27.443332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9c62e-0c45-4752-9516-0b34ae1d9b27 admin admin] PATH_INFO: `/v3/users/6490654f4f4543e6a1f51cfaf543176c` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:27.444721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb9c62e-0c45-4752-9516-0b34ae1d9b27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6490654f4f4543e6a1f51cfaf543176c)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:27.447161 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41161dcd-6ec5-4878-9acc-c9a0c4f3b74b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:27.453108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41161dcd-6ec5-4878-9acc-c9a0c4f3b74b service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:27.453271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb9c62e-0c45-4752-9516-0b34ae1d9b27 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:27.453364 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41161dcd-6ec5-4878-9acc-c9a0c4f3b74b service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:27.453364 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41161dcd-6ec5-4878-9acc-c9a0c4f3b74b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:27.462541 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41161dcd-6ec5-4878-9acc-c9a0c4f3b74b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:27.464109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41161dcd-6ec5-4878-9acc-c9a0c4f3b74b service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:27.474847 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1017/2028] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:34:27 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:34:27.515390 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-0eb9c62e-0c45-4752-9516-0b34ae1d9b27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6490654f4f4543e6a1f51cfaf543176c'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:27.515390 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-0eb9c62e-0c45-4752-9516-0b34ae1d9b27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6490654f4f4543e6a1f51cfaf543176c'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:27.525699 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-0eb9c62e-0c45-4752-9516-0b34ae1d9b27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6490654f4f4543e6a1f51cfaf543176c'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:27.541043 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-0eb9c62e-0c45-4752-9516-0b34ae1d9b27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6490654f4f4543e6a1f51cfaf543176c'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:27.550475 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-0eb9c62e-0c45-4752-9516-0b34ae1d9b27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6490654f4f4543e6a1f51cfaf543176c'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:27.558733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1012/2029] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:34:27 2022] DELETE /identity/v3/users/6490654f4f4543e6a1f51cfaf543176c => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:27.565096 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41161dcd-6ec5-4878-9acc-c9a0c4f3b74b service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:27.576671 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41161dcd-6ec5-4878-9acc-c9a0c4f3b74b service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:27.640417 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334434d-2354-4fd2-998e-a8f6cdbe36ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:27.641271 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334434d-2354-4fd2-998e-a8f6cdbe36ff service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:27.641568 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334434d-2354-4fd2-998e-a8f6cdbe36ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:27.641856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334434d-2354-4fd2-998e-a8f6cdbe36ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:27.644216 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-6334434d-2354-4fd2-998e-a8f6cdbe36ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:27.728678 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334434d-2354-4fd2-998e-a8f6cdbe36ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:27.730237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6334434d-2354-4fd2-998e-a8f6cdbe36ff service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:27.748763 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1018/2030] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:34:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:34:27.760449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9c62e-0c45-4752-9516-0b34ae1d9b27 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:27.762165 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-0eb9c62e-0c45-4752-9516-0b34ae1d9b27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:27.857283 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9c62e-0c45-4752-9516-0b34ae1d9b27 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:27.881736 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6cee0-e39c-46bb-815d-3ed5ad0b3b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:27.883601 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6cee0-e39c-46bb-815d-3ed5ad0b3b2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:27.883946 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6cee0-e39c-46bb-815d-3ed5ad0b3b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:27.884436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6cee0-e39c-46bb-815d-3ed5ad0b3b2e admin admin] PATH_INFO: `/v3/projects/5add1b56848d44eda7b0dae0b34bb67e` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:27.886185 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c6cee0-e39c-46bb-815d-3ed5ad0b3b2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5add1b56848d44eda7b0dae0b34bb67e)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:27.888372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c6cee0-e39c-46bb-815d-3ed5ad0b3b2e admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:27.953219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-c5c6cee0-e39c-46bb-815d-3ed5ad0b3b2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5add1b56848d44eda7b0dae0b34bb67e', 'request_id': 'req-c5c6cee0-e39c-46bb-815d-3ed5ad0b3b2e'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:27.967198 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-c5c6cee0-e39c-46bb-815d-3ed5ad0b3b2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5add1b56848d44eda7b0dae0b34bb67e', 'request_id': 'req-c5c6cee0-e39c-46bb-815d-3ed5ad0b3b2e'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:27.978651 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-c5c6cee0-e39c-46bb-815d-3ed5ad0b3b2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5add1b56848d44eda7b0dae0b34bb67e', 'request_id': 'req-c5c6cee0-e39c-46bb-815d-3ed5ad0b3b2e'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:27.992540 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-c5c6cee0-e39c-46bb-815d-3ed5ad0b3b2e admin admin] The token cache is being invalidate because project 5add1b56848d44eda7b0dae0b34bb67e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:34:27.993134 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-c5c6cee0-e39c-46bb-815d-3ed5ad0b3b2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:27.994009 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-c5c6cee0-e39c-46bb-815d-3ed5ad0b3b2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:27.996211 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1013/2031] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:34:27 2022] DELETE /identity/v3/projects/5add1b56848d44eda7b0dae0b34bb67e => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:28.001490 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334434d-2354-4fd2-998e-a8f6cdbe36ff service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:28.013567 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334434d-2354-4fd2-998e-a8f6cdbe36ff service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:28.096074 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829518f9-6748-4647-bc19-a83b78b2a7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:28.097491 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829518f9-6748-4647-bc19-a83b78b2a7f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:28.097958 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829518f9-6748-4647-bc19-a83b78b2a7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:28.098540 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829518f9-6748-4647-bc19-a83b78b2a7f9 admin admin] PATH_INFO: `/v3/users/a2a9d4a7641a42bfabef15189732e059` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:28.100320 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829518f9-6748-4647-bc19-a83b78b2a7f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2a9d4a7641a42bfabef15189732e059)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:28.104330 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829518f9-6748-4647-bc19-a83b78b2a7f9 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:28.155179 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:28.171668 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-829518f9-6748-4647-bc19-a83b78b2a7f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2a9d4a7641a42bfabef15189732e059'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:28.187194 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-829518f9-6748-4647-bc19-a83b78b2a7f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2a9d4a7641a42bfabef15189732e059'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:28.196834 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-829518f9-6748-4647-bc19-a83b78b2a7f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2a9d4a7641a42bfabef15189732e059'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:28.197498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-829518f9-6748-4647-bc19-a83b78b2a7f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2a9d4a7641a42bfabef15189732e059'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:28.198140 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-829518f9-6748-4647-bc19-a83b78b2a7f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2a9d4a7641a42bfabef15189732e059'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:28.213490 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1019/2032] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:34:28 2022] DELETE /identity/v3/users/a2a9d4a7641a42bfabef15189732e059 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:28.490964 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6cee0-e39c-46bb-815d-3ed5ad0b3b2e admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:28.493783 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-c5c6cee0-e39c-46bb-815d-3ed5ad0b3b2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:28.508059 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6cee0-e39c-46bb-815d-3ed5ad0b3b2e admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:28.513953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:28.593035 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59d158-7d0e-4121-b44e-ff667c772869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:28.593947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59d158-7d0e-4121-b44e-ff667c772869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:28.594230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59d158-7d0e-4121-b44e-ff667c772869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:28.594507 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59d158-7d0e-4121-b44e-ff667c772869 admin admin] PATH_INFO: `/v3/projects/866958591e4d49b3b2b6402dbd7db49a` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:28.595923 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e59d158-7d0e-4121-b44e-ff667c772869 admin admin] RBAC: Authorizing `identity:delete_project(project_id=866958591e4d49b3b2b6402dbd7db49a)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:28.597895 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e59d158-7d0e-4121-b44e-ff667c772869 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:28.653297 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2e59d158-7d0e-4121-b44e-ff667c772869 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '866958591e4d49b3b2b6402dbd7db49a', 'request_id': 'req-2e59d158-7d0e-4121-b44e-ff667c772869'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:28.663379 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2e59d158-7d0e-4121-b44e-ff667c772869 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '866958591e4d49b3b2b6402dbd7db49a', 'request_id': 'req-2e59d158-7d0e-4121-b44e-ff667c772869'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:28.670917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2e59d158-7d0e-4121-b44e-ff667c772869 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '866958591e4d49b3b2b6402dbd7db49a', 'request_id': 'req-2e59d158-7d0e-4121-b44e-ff667c772869'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:28.685086 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2e59d158-7d0e-4121-b44e-ff667c772869 admin admin] The token cache is being invalidate because project 866958591e4d49b3b2b6402dbd7db49a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:34:28.685389 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2e59d158-7d0e-4121-b44e-ff667c772869 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:28.686083 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2e59d158-7d0e-4121-b44e-ff667c772869 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:28.687699 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1014/2033] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:34:28 2022] DELETE /identity/v3/projects/866958591e4d49b3b2b6402dbd7db49a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:28.692685 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec1c696-1661-4cb5-8ace-82ad8c59aa53 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:28.692979 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec1c696-1661-4cb5-8ace-82ad8c59aa53 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:28.693266 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec1c696-1661-4cb5-8ace-82ad8c59aa53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:28.714483 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-2ec1c696-1661-4cb5-8ace-82ad8c59aa53 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:28.715711 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-2ec1c696-1661-4cb5-8ace-82ad8c59aa53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:28.779040 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1020/2034] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:28.787011 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb32b1-8a48-4fa8-9ed9-25efd8c60b88 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:28.787307 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb32b1-8a48-4fa8-9ed9-25efd8c60b88 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:28.787595 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb32b1-8a48-4fa8-9ed9-25efd8c60b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:28.807163 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-dafb32b1-8a48-4fa8-9ed9-25efd8c60b88 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:28.808438 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-dafb32b1-8a48-4fa8-9ed9-25efd8c60b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:28.816532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1015/2035] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:28.820947 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec1c696-1661-4cb5-8ace-82ad8c59aa53 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:28.832336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec1c696-1661-4cb5-8ace-82ad8c59aa53 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:28.837493 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f448f-2d87-41af-9007-bac8ddf2c276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:28.838178 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f448f-2d87-41af-9007-bac8ddf2c276 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:28.838481 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f448f-2d87-41af-9007-bac8ddf2c276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:28.838755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f448f-2d87-41af-9007-bac8ddf2c276 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:28.839477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f448f-2d87-41af-9007-bac8ddf2c276 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:34:28.839783 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f448f-2d87-41af-9007-bac8ddf2c276 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:28.842088 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f448f-2d87-41af-9007-bac8ddf2c276 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:28.855246 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1021/2036] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:34:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:34:28.859489 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb32b1-8a48-4fa8-9ed9-25efd8c60b88 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:28.871289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb32b1-8a48-4fa8-9ed9-25efd8c60b88 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:28.876003 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c3b17a-ac3e-441e-b95c-103b16048521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:28.876720 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3b17a-ac3e-441e-b95c-103b16048521 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:28.876996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3b17a-ac3e-441e-b95c-103b16048521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:28.877277 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3b17a-ac3e-441e-b95c-103b16048521 admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:28.877974 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c3b17a-ac3e-441e-b95c-103b16048521 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:28.879903 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c3b17a-ac3e-441e-b95c-103b16048521 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:28.913771 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1016/2037] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:34:28 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:34:28.918716 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f448f-2d87-41af-9007-bac8ddf2c276 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:28.928922 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f448f-2d87-41af-9007-bac8ddf2c276 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:28.990971 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f518ba0-6657-4700-b6ce-927de946ea0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:28.991661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f518ba0-6657-4700-b6ce-927de946ea0c admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:28.992269 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f518ba0-6657-4700-b6ce-927de946ea0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:28.992385 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f518ba0-6657-4700-b6ce-927de946ea0c admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:28.993082 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f518ba0-6657-4700-b6ce-927de946ea0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:28.995101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f518ba0-6657-4700-b6ce-927de946ea0c admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:29.033344 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1022/2038] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:34:28 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:34:29.037929 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c3b17a-ac3e-441e-b95c-103b16048521 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:29.068758 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c3b17a-ac3e-441e-b95c-103b16048521 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:29.080306 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc939c-e451-4795-93b6-c2e3d63f5a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:29.081150 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc939c-e451-4795-93b6-c2e3d63f5a5d admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:29.081524 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc939c-e451-4795-93b6-c2e3d63f5a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:29.081823 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bc939c-e451-4795-93b6-c2e3d63f5a5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:29.082653 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bc939c-e451-4795-93b6-c2e3d63f5a5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:29.084377 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bc939c-e451-4795-93b6-c2e3d63f5a5d admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:29.099438 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1017/2039] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:34:29.102000 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f518ba0-6657-4700-b6ce-927de946ea0c admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:29.118606 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f518ba0-6657-4700-b6ce-927de946ea0c admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:29.125675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13356878-d3ba-4eb9-8af7-e6ab84f5ddbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:29.126362 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13356878-d3ba-4eb9-8af7-e6ab84f5ddbf admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:29.126645 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13356878-d3ba-4eb9-8af7-e6ab84f5ddbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:29.126923 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13356878-d3ba-4eb9-8af7-e6ab84f5ddbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:29.127680 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13356878-d3ba-4eb9-8af7-e6ab84f5ddbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:29.130030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13356878-d3ba-4eb9-8af7-e6ab84f5ddbf admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:29.155121 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1023/2040] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:34:29.159891 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc939c-e451-4795-93b6-c2e3d63f5a5d admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:29.173902 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bc939c-e451-4795-93b6-c2e3d63f5a5d admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:29.186067 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa5cb6-7e35-404c-832b-037406337154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:29.187035 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa5cb6-7e35-404c-832b-037406337154 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:29.187328 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa5cb6-7e35-404c-832b-037406337154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:29.188581 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa5cb6-7e35-404c-832b-037406337154 admin admin] PATH_INFO: `/v3/projects/70096c9a67564164a58780f6d5e9b227/users/620fa115db104a8d82f7238a379c736f/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:29.215492 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa5cb6-7e35-404c-832b-037406337154 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70096c9a67564164a58780f6d5e9b227, user_id=620fa115db104a8d82f7238a379c736f, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:29.221870 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa5cb6-7e35-404c-832b-037406337154 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:29.256657 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1018/2041] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:34:29 2022] PUT /identity/v3/projects/70096c9a67564164a58780f6d5e9b227/users/620fa115db104a8d82f7238a379c736f/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:34:29.267242 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac3fd4-3484-4715-81ae-9d465dd99d4a None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:29.267574 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac3fd4-3484-4715-81ae-9d465dd99d4a None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:29.267956 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac3fd4-3484-4715-81ae-9d465dd99d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:29.314950 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:29.319886 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-93ac3fd4-3484-4715-81ae-9d465dd99d4a None None] MFA Rules not processed for user `620fa115db104a8d82f7238a379c736f`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:29.323562 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-93ac3fd4-3484-4715-81ae-9d465dd99d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:29.439768 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1024/2042] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:29.443955 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa5cb6-7e35-404c-832b-037406337154 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:29.522114 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:29.585813 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa5cb6-7e35-404c-832b-037406337154 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:29.593173 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf00970-7762-48bf-8088-d6f8b23fa770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:29.594424 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf00970-7762-48bf-8088-d6f8b23fa770 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:29.594787 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf00970-7762-48bf-8088-d6f8b23fa770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:29.595190 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf00970-7762-48bf-8088-d6f8b23fa770 admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:29.596626 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf00970-7762-48bf-8088-d6f8b23fa770 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:29.598668 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf00970-7762-48bf-8088-d6f8b23fa770 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:29.638402 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1019/2043] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:34:29 2022] POST /identity/v3/projects => generated 383 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:34:29.643380 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac3fd4-3484-4715-81ae-9d465dd99d4a None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:29.676487 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac3fd4-3484-4715-81ae-9d465dd99d4a None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:29.751580 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b70880-8bb7-447e-a2bd-149b1501c56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:29.752370 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b70880-8bb7-447e-a2bd-149b1501c56f admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:29.752654 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b70880-8bb7-447e-a2bd-149b1501c56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:29.752926 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b70880-8bb7-447e-a2bd-149b1501c56f admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:29.753635 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b70880-8bb7-447e-a2bd-149b1501c56f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:29.755789 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b70880-8bb7-447e-a2bd-149b1501c56f admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:29.785152 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1025/2044] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:34:29 2022] POST /identity/v3/users => generated 298 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:34:29.787927 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf00970-7762-48bf-8088-d6f8b23fa770 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:29.806786 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf00970-7762-48bf-8088-d6f8b23fa770 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:29.812377 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e38d2-f36a-4b09-ba00-9a31e0f580bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:29.812838 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e38d2-f36a-4b09-ba00-9a31e0f580bd admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:29.813213 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e38d2-f36a-4b09-ba00-9a31e0f580bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:29.813564 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e38d2-f36a-4b09-ba00-9a31e0f580bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:29.814382 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2e38d2-f36a-4b09-ba00-9a31e0f580bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:29.816080 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2e38d2-f36a-4b09-ba00-9a31e0f580bd admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:29.831292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1020/2045] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:34:29.837148 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b70880-8bb7-447e-a2bd-149b1501c56f admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:29.849867 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b70880-8bb7-447e-a2bd-149b1501c56f admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:29.856001 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debcc1f6-b6b8-4cf2-be7f-cd82fec30e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:29.857031 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcc1f6-b6b8-4cf2-be7f-cd82fec30e2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:29.857289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcc1f6-b6b8-4cf2-be7f-cd82fec30e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:29.857568 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcc1f6-b6b8-4cf2-be7f-cd82fec30e2a admin admin] PATH_INFO: `/v3/projects/03083278661a4cd69237c223badba3f2/users/4528bda94cc547489f062801cc8fd491/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:29.888208 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debcc1f6-b6b8-4cf2-be7f-cd82fec30e2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=03083278661a4cd69237c223badba3f2, user_id=4528bda94cc547489f062801cc8fd491, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:29.891190 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debcc1f6-b6b8-4cf2-be7f-cd82fec30e2a admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:29.913332 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1026/2046] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:34:29 2022] PUT /identity/v3/projects/03083278661a4cd69237c223badba3f2/users/4528bda94cc547489f062801cc8fd491/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:34:29.918166 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e38d2-f36a-4b09-ba00-9a31e0f580bd admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:29.929655 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e38d2-f36a-4b09-ba00-9a31e0f580bd admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:29.999740 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb487c-126d-4ab2-af19-a24cb0cf9028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:30.000600 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb487c-126d-4ab2-af19-a24cb0cf9028 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:30.001034 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb487c-126d-4ab2-af19-a24cb0cf9028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:30.001448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb487c-126d-4ab2-af19-a24cb0cf9028 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:30.002328 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb487c-126d-4ab2-af19-a24cb0cf9028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:30.004238 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb487c-126d-4ab2-af19-a24cb0cf9028 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:30.016890 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1021/2047] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:34:30.021237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debcc1f6-b6b8-4cf2-be7f-cd82fec30e2a admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:30.036424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debcc1f6-b6b8-4cf2-be7f-cd82fec30e2a admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:30.044079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b86eb4-45e6-4877-9a1c-ede86c952e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:30.044955 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b86eb4-45e6-4877-9a1c-ede86c952e11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:30.045229 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b86eb4-45e6-4877-9a1c-ede86c952e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:30.045500 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b86eb4-45e6-4877-9a1c-ede86c952e11 admin admin] PATH_INFO: `/v3/projects/03083278661a4cd69237c223badba3f2/users/4528bda94cc547489f062801cc8fd491/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:30.047686 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b86eb4-45e6-4877-9a1c-ede86c952e11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03083278661a4cd69237c223badba3f2, user_id=4528bda94cc547489f062801cc8fd491, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:30.050781 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b86eb4-45e6-4877-9a1c-ede86c952e11 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:30.062606 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1027/2048] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:34:30 2022] PUT /identity/v3/projects/03083278661a4cd69237c223badba3f2/users/4528bda94cc547489f062801cc8fd491/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:30.073869 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9377621-d34e-4567-931f-807edc409af2 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:30.074866 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9377621-d34e-4567-931f-807edc409af2 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:30.074866 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9377621-d34e-4567-931f-807edc409af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:30.106239 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-d9377621-d34e-4567-931f-807edc409af2 None None] MFA Rules not processed for user `4528bda94cc547489f062801cc8fd491`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:30.107615 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-d9377621-d34e-4567-931f-807edc409af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:30.198980 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1022/2049] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:30 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:34:30.210417 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b86eb4-45e6-4877-9a1c-ede86c952e11 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:30.212342 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a3b86eb4-45e6-4877-9a1c-ede86c952e11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:30.306913 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b86eb4-45e6-4877-9a1c-ede86c952e11 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:30.311338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1aecfd-afa8-4339-a01e-c828ba65dc24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:30.312121 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1aecfd-afa8-4339-a01e-c828ba65dc24 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:30.312531 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1aecfd-afa8-4339-a01e-c828ba65dc24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:30.312881 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1aecfd-afa8-4339-a01e-c828ba65dc24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:30.325062 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:30.403079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1aecfd-afa8-4339-a01e-c828ba65dc24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:30.405059 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1aecfd-afa8-4339-a01e-c828ba65dc24 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:30.415739 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1028/2050] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:34:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:34:30.888370 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9377621-d34e-4567-931f-807edc409af2 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:30.894150 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-d9377621-d34e-4567-931f-807edc409af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:30.973256 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:31.035483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9377621-d34e-4567-931f-807edc409af2 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:31.041396 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f4e45-8009-4a27-91c9-54663d71ca97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:31.042386 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f4e45-8009-4a27-91c9-54663d71ca97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:31.042765 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f4e45-8009-4a27-91c9-54663d71ca97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:31.043114 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f4e45-8009-4a27-91c9-54663d71ca97 admin admin] PATH_INFO: `/v3/users/884b5ede502c48459d2770dedcb7aacc` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:31.044531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6f4e45-8009-4a27-91c9-54663d71ca97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=884b5ede502c48459d2770dedcb7aacc)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:31.046754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6f4e45-8009-4a27-91c9-54663d71ca97 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:31.136735 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bf6f4e45-8009-4a27-91c9-54663d71ca97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '884b5ede502c48459d2770dedcb7aacc'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:31.137492 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bf6f4e45-8009-4a27-91c9-54663d71ca97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '884b5ede502c48459d2770dedcb7aacc'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:31.145524 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bf6f4e45-8009-4a27-91c9-54663d71ca97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '884b5ede502c48459d2770dedcb7aacc'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:31.168154 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bf6f4e45-8009-4a27-91c9-54663d71ca97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '884b5ede502c48459d2770dedcb7aacc'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:31.176719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bf6f4e45-8009-4a27-91c9-54663d71ca97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '884b5ede502c48459d2770dedcb7aacc'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:31.190215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1023/2051] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:34:30 2022] DELETE /identity/v3/users/884b5ede502c48459d2770dedcb7aacc => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:31.192239 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c53dc-3717-4375-a74c-de1d2d14262f None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:31.192695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c53dc-3717-4375-a74c-de1d2d14262f None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:31.192814 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c53dc-3717-4375-a74c-de1d2d14262f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:31.213772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-1b2c53dc-3717-4375-a74c-de1d2d14262f None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:31.217438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1b2c53dc-3717-4375-a74c-de1d2d14262f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:31.302209 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1029/2052] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:31.313936 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f4e45-8009-4a27-91c9-54663d71ca97 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:31.315880 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-bf6f4e45-8009-4a27-91c9-54663d71ca97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:31.422634 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f4e45-8009-4a27-91c9-54663d71ca97 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:31.444373 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1952fa-c1d3-446b-a1ac-389324df1d45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:31.445463 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1952fa-c1d3-446b-a1ac-389324df1d45 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:31.445857 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1952fa-c1d3-446b-a1ac-389324df1d45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:31.446441 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1952fa-c1d3-446b-a1ac-389324df1d45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:31.462841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1952fa-c1d3-446b-a1ac-389324df1d45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:31.464563 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1952fa-c1d3-446b-a1ac-389324df1d45 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:31.479941 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1024/2053] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:34:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:34:31.825082 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c53dc-3717-4375-a74c-de1d2d14262f None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:31.826843 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1b2c53dc-3717-4375-a74c-de1d2d14262f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:31.840016 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c53dc-3717-4375-a74c-de1d2d14262f None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:31.845143 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391e9e6-37b5-4120-98a5-e3af15ba89fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:31.846033 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391e9e6-37b5-4120-98a5-e3af15ba89fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:31.846319 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391e9e6-37b5-4120-98a5-e3af15ba89fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:31.846605 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391e9e6-37b5-4120-98a5-e3af15ba89fc admin admin] PATH_INFO: `/v3/projects/60b0caf397e04eb88297de46e7f556e0` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:31.847822 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4391e9e6-37b5-4120-98a5-e3af15ba89fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=60b0caf397e04eb88297de46e7f556e0)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:31.849945 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4391e9e6-37b5-4120-98a5-e3af15ba89fc admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:31.908889 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4391e9e6-37b5-4120-98a5-e3af15ba89fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60b0caf397e04eb88297de46e7f556e0', 'request_id': 'req-4391e9e6-37b5-4120-98a5-e3af15ba89fc'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:31.916254 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4391e9e6-37b5-4120-98a5-e3af15ba89fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60b0caf397e04eb88297de46e7f556e0', 'request_id': 'req-4391e9e6-37b5-4120-98a5-e3af15ba89fc'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:31.923659 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4391e9e6-37b5-4120-98a5-e3af15ba89fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60b0caf397e04eb88297de46e7f556e0', 'request_id': 'req-4391e9e6-37b5-4120-98a5-e3af15ba89fc'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:31.938861 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4391e9e6-37b5-4120-98a5-e3af15ba89fc admin admin] The token cache is being invalidate because project 60b0caf397e04eb88297de46e7f556e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:34:31.939155 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4391e9e6-37b5-4120-98a5-e3af15ba89fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:31.939853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4391e9e6-37b5-4120-98a5-e3af15ba89fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:31.941767 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1030/2054] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:34:31 2022] DELETE /identity/v3/projects/60b0caf397e04eb88297de46e7f556e0 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:31.949268 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd362ba1-c3d0-4089-8c6b-55c1092f6074 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:31.949727 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd362ba1-c3d0-4089-8c6b-55c1092f6074 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:31.950150 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd362ba1-c3d0-4089-8c6b-55c1092f6074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:31.972957 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-cd362ba1-c3d0-4089-8c6b-55c1092f6074 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:31.974218 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-cd362ba1-c3d0-4089-8c6b-55c1092f6074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:32.042175 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:32.059594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1025/2055] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:32.067194 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a1334-81f3-40b5-980b-f626406baeef None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:32.067562 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a1334-81f3-40b5-980b-f626406baeef None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:32.067788 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a1334-81f3-40b5-980b-f626406baeef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:32.087643 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a38a1334-81f3-40b5-980b-f626406baeef None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:32.088889 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a38a1334-81f3-40b5-980b-f626406baeef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:32.098296 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1031/2056] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:34:32.103975 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd362ba1-c3d0-4089-8c6b-55c1092f6074 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:32.114884 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd362ba1-c3d0-4089-8c6b-55c1092f6074 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:32.120733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5ebdf-78ad-42b1-8eb8-366e88cd780c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:32.121907 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5ebdf-78ad-42b1-8eb8-366e88cd780c admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:32.121907 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5ebdf-78ad-42b1-8eb8-366e88cd780c admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:32.122122 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d5ebdf-78ad-42b1-8eb8-366e88cd780c admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:32.122685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d5ebdf-78ad-42b1-8eb8-366e88cd780c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:34:32.123000 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d5ebdf-78ad-42b1-8eb8-366e88cd780c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:32.124681 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d5ebdf-78ad-42b1-8eb8-366e88cd780c admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:32.138756 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1026/2057] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:34:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:34:32.143410 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a1334-81f3-40b5-980b-f626406baeef None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:32.153272 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a1334-81f3-40b5-980b-f626406baeef None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:32.159460 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2997c-3f65-4655-b7d3-dc67933e853b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:32.160305 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2997c-3f65-4655-b7d3-dc67933e853b admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:32.160771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2997c-3f65-4655-b7d3-dc67933e853b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:32.161091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c2997c-3f65-4655-b7d3-dc67933e853b admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:32.162046 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c2997c-3f65-4655-b7d3-dc67933e853b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:32.164736 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c2997c-3f65-4655-b7d3-dc67933e853b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:32.195679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1032/2058] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:34:32 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:34:32.199750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5ebdf-78ad-42b1-8eb8-366e88cd780c admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:32.215162 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d5ebdf-78ad-42b1-8eb8-366e88cd780c admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:32.314951 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f1627-c1ec-4380-942a-277167a5847c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:32.315644 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f1627-c1ec-4380-942a-277167a5847c admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:32.315933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f1627-c1ec-4380-942a-277167a5847c admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:32.316266 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f1627-c1ec-4380-942a-277167a5847c admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:32.316953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688f1627-c1ec-4380-942a-277167a5847c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:32.318939 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688f1627-c1ec-4380-942a-277167a5847c admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:32.342208 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1027/2059] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:34:32 2022] POST /identity/v3/users => generated 287 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:34:32.346881 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2997c-3f65-4655-b7d3-dc67933e853b admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:32.355139 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:32.359404 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c2997c-3f65-4655-b7d3-dc67933e853b admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:32.379111 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9321b941-dcfc-40bf-aab1-9376e7e5bd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:32.380249 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321b941-dcfc-40bf-aab1-9376e7e5bd8c admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:32.380745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321b941-dcfc-40bf-aab1-9376e7e5bd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:32.380997 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9321b941-dcfc-40bf-aab1-9376e7e5bd8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:32.382238 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9321b941-dcfc-40bf-aab1-9376e7e5bd8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:32.387122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9321b941-dcfc-40bf-aab1-9376e7e5bd8c admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:32.401057 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1033/2060] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:34:32.405258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f1627-c1ec-4380-942a-277167a5847c admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:32.416726 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f1627-c1ec-4380-942a-277167a5847c admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:32.421540 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a57be5-ba5e-4d0a-939c-0609c17cdc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:32.422199 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a57be5-ba5e-4d0a-939c-0609c17cdc18 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:32.422483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a57be5-ba5e-4d0a-939c-0609c17cdc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:32.422762 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a57be5-ba5e-4d0a-939c-0609c17cdc18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:32.423497 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a57be5-ba5e-4d0a-939c-0609c17cdc18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:32.425149 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a57be5-ba5e-4d0a-939c-0609c17cdc18 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:32.439906 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1028/2061] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:34:32.444331 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9321b941-dcfc-40bf-aab1-9376e7e5bd8c admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:32.455420 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9321b941-dcfc-40bf-aab1-9376e7e5bd8c admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:32.460156 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f194e055-92d4-4508-81cc-cb4d3058ca24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:32.461712 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194e055-92d4-4508-81cc-cb4d3058ca24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:32.462031 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194e055-92d4-4508-81cc-cb4d3058ca24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:32.462371 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194e055-92d4-4508-81cc-cb4d3058ca24 admin admin] PATH_INFO: `/v3/projects/087539c0e4c349c8a06ea352bcc4f3f7/users/69bd1f290f3744d4b4d1f873f38efd82/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:32.478020 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f194e055-92d4-4508-81cc-cb4d3058ca24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=087539c0e4c349c8a06ea352bcc4f3f7, user_id=69bd1f290f3744d4b4d1f873f38efd82, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:32.480749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f194e055-92d4-4508-81cc-cb4d3058ca24 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:32.495822 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1034/2062] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:34:32 2022] PUT /identity/v3/projects/087539c0e4c349c8a06ea352bcc4f3f7/users/69bd1f290f3744d4b4d1f873f38efd82/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:32.506380 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74eeea-3669-4649-aa52-9cf125b7a7d0 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:32.506677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74eeea-3669-4649-aa52-9cf125b7a7d0 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:32.506966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe74eeea-3669-4649-aa52-9cf125b7a7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:32.536493 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-fe74eeea-3669-4649-aa52-9cf125b7a7d0 None None] MFA Rules not processed for user `69bd1f290f3744d4b4d1f873f38efd82`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:32.537730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-fe74eeea-3669-4649-aa52-9cf125b7a7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:32.610611 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1029/2063] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:32.623282 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f194e055-92d4-4508-81cc-cb4d3058ca24 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:32.625026 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-f194e055-92d4-4508-81cc-cb4d3058ca24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:32.687379 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f194e055-92d4-4508-81cc-cb4d3058ca24 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:32.691250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b65791b-7856-4317-832c-a230466bfd3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:32.691982 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65791b-7856-4317-832c-a230466bfd3a service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:32.692392 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65791b-7856-4317-832c-a230466bfd3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:32.692743 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65791b-7856-4317-832c-a230466bfd3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:32.702682 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b65791b-7856-4317-832c-a230466bfd3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:32.704261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b65791b-7856-4317-832c-a230466bfd3a service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:32.715518 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1035/2064] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:34:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:34:40.058557 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe74eeea-3669-4649-aa52-9cf125b7a7d0 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:40.080274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe74eeea-3669-4649-aa52-9cf125b7a7d0 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:40.084065 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e033aa70-8b71-4466-87af-05f6277c9317 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:40.084751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033aa70-8b71-4466-87af-05f6277c9317 service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:40.085050 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033aa70-8b71-4466-87af-05f6277c9317 service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:40.086070 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033aa70-8b71-4466-87af-05f6277c9317 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:40.087704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-e033aa70-8b71-4466-87af-05f6277c9317 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:40.125695 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:40.203570 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e033aa70-8b71-4466-87af-05f6277c9317 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:40.205070 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e033aa70-8b71-4466-87af-05f6277c9317 service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:40.218448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1030/2065] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:34:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:34:42.713959 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=39,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:47.361123 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b65791b-7856-4317-832c-a230466bfd3a service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:47.362679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3b65791b-7856-4317-832c-a230466bfd3a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:47.501847 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b65791b-7856-4317-832c-a230466bfd3a service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:47.507631 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4aa322-708a-47d1-8c35-cbacb0cbed60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:47.508569 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4aa322-708a-47d1-8c35-cbacb0cbed60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:47.508852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4aa322-708a-47d1-8c35-cbacb0cbed60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:47.509128 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4aa322-708a-47d1-8c35-cbacb0cbed60 admin admin] PATH_INFO: `/v3/users/620fa115db104a8d82f7238a379c736f` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:47.511575 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4aa322-708a-47d1-8c35-cbacb0cbed60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=620fa115db104a8d82f7238a379c736f)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:47.512349 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4aa322-708a-47d1-8c35-cbacb0cbed60 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:47.564037 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ff4aa322-708a-47d1-8c35-cbacb0cbed60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '620fa115db104a8d82f7238a379c736f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:47.579743 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ff4aa322-708a-47d1-8c35-cbacb0cbed60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '620fa115db104a8d82f7238a379c736f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:47.588618 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ff4aa322-708a-47d1-8c35-cbacb0cbed60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '620fa115db104a8d82f7238a379c736f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:47.591441 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ff4aa322-708a-47d1-8c35-cbacb0cbed60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '620fa115db104a8d82f7238a379c736f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:47.592137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ff4aa322-708a-47d1-8c35-cbacb0cbed60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '620fa115db104a8d82f7238a379c736f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:47.615534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1036/2066] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:34:47 2022] DELETE /identity/v3/users/620fa115db104a8d82f7238a379c736f => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:47.621122 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05e9f2-e8d2-4a24-96a7-fa37a1e4c769 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:47.621578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05e9f2-e8d2-4a24-96a7-fa37a1e4c769 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:47.622313 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05e9f2-e8d2-4a24-96a7-fa37a1e4c769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:47.656433 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-dd05e9f2-e8d2-4a24-96a7-fa37a1e4c769 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:47.657767 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-dd05e9f2-e8d2-4a24-96a7-fa37a1e4c769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:47.727847 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1031/2067] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:47.737663 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4aa322-708a-47d1-8c35-cbacb0cbed60 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:47.739068 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ff4aa322-708a-47d1-8c35-cbacb0cbed60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:47.778453 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:47.847108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4aa322-708a-47d1-8c35-cbacb0cbed60 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:47.847108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d4caa-7add-4912-9d61-3d1977f975b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:47.847108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d4caa-7add-4912-9d61-3d1977f975b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:47.847108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d4caa-7add-4912-9d61-3d1977f975b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:47.847565 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d4caa-7add-4912-9d61-3d1977f975b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:47.855624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04d4caa-7add-4912-9d61-3d1977f975b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:47.857111 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04d4caa-7add-4912-9d61-3d1977f975b8 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:47.868786 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1037/2068] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:34:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:34:48.151474 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd05e9f2-e8d2-4a24-96a7-fa37a1e4c769 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:48.153081 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-dd05e9f2-e8d2-4a24-96a7-fa37a1e4c769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:48.174430 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd05e9f2-e8d2-4a24-96a7-fa37a1e4c769 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:48.177874 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb72ad-e62c-4629-a5b1-a22dffff4c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:48.178841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb72ad-e62c-4629-a5b1-a22dffff4c32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:48.179620 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb72ad-e62c-4629-a5b1-a22dffff4c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:48.180341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb72ad-e62c-4629-a5b1-a22dffff4c32 admin admin] PATH_INFO: `/v3/projects/70096c9a67564164a58780f6d5e9b227` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:48.181620 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb72ad-e62c-4629-a5b1-a22dffff4c32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70096c9a67564164a58780f6d5e9b227)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:48.183584 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb72ad-e62c-4629-a5b1-a22dffff4c32 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:48.240946 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-4dbb72ad-e62c-4629-a5b1-a22dffff4c32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70096c9a67564164a58780f6d5e9b227', 'request_id': 'req-4dbb72ad-e62c-4629-a5b1-a22dffff4c32'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:48.249625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-4dbb72ad-e62c-4629-a5b1-a22dffff4c32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70096c9a67564164a58780f6d5e9b227', 'request_id': 'req-4dbb72ad-e62c-4629-a5b1-a22dffff4c32'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:48.256745 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-4dbb72ad-e62c-4629-a5b1-a22dffff4c32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70096c9a67564164a58780f6d5e9b227', 'request_id': 'req-4dbb72ad-e62c-4629-a5b1-a22dffff4c32'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:48.269656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-4dbb72ad-e62c-4629-a5b1-a22dffff4c32 admin admin] The token cache is being invalidate because project 70096c9a67564164a58780f6d5e9b227 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:34:48.269946 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-4dbb72ad-e62c-4629-a5b1-a22dffff4c32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:48.270614 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-4dbb72ad-e62c-4629-a5b1-a22dffff4c32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:48.272258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1032/2069] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:34:48 2022] DELETE /identity/v3/projects/70096c9a67564164a58780f6d5e9b227 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:48.276772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d4caa-7add-4912-9d61-3d1977f975b8 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:48.289799 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d4caa-7add-4912-9d61-3d1977f975b8 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:48.350455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7bd01-e5fb-4150-87c1-f87af537215c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:48.351335 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7bd01-e5fb-4150-87c1-f87af537215c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:48.351803 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7bd01-e5fb-4150-87c1-f87af537215c admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:48.352085 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7bd01-e5fb-4150-87c1-f87af537215c admin admin] PATH_INFO: `/v3/users/4528bda94cc547489f062801cc8fd491` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:48.353378 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef7bd01-e5fb-4150-87c1-f87af537215c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4528bda94cc547489f062801cc8fd491)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:48.355297 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef7bd01-e5fb-4150-87c1-f87af537215c admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:48.408845 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4ef7bd01-e5fb-4150-87c1-f87af537215c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4528bda94cc547489f062801cc8fd491'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:48.423242 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4ef7bd01-e5fb-4150-87c1-f87af537215c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4528bda94cc547489f062801cc8fd491'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:48.434294 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4ef7bd01-e5fb-4150-87c1-f87af537215c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4528bda94cc547489f062801cc8fd491'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:48.435132 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4ef7bd01-e5fb-4150-87c1-f87af537215c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4528bda94cc547489f062801cc8fd491'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:48.435749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4ef7bd01-e5fb-4150-87c1-f87af537215c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4528bda94cc547489f062801cc8fd491'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:48.451150 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1038/2070] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:34:48 2022] DELETE /identity/v3/users/4528bda94cc547489f062801cc8fd491 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:48.757497 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb72ad-e62c-4629-a5b1-a22dffff4c32 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:48.759032 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-4dbb72ad-e62c-4629-a5b1-a22dffff4c32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:48.772761 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb72ad-e62c-4629-a5b1-a22dffff4c32 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:48.790212 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:48.838557 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4cb53-0511-400b-b5a1-c0019e24c530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:48.839429 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4cb53-0511-400b-b5a1-c0019e24c530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:48.839715 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4cb53-0511-400b-b5a1-c0019e24c530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:48.839994 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4cb53-0511-400b-b5a1-c0019e24c530 admin admin] PATH_INFO: `/v3/projects/03083278661a4cd69237c223badba3f2` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:48.841220 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb4cb53-0511-400b-b5a1-c0019e24c530 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03083278661a4cd69237c223badba3f2)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:48.843139 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb4cb53-0511-400b-b5a1-c0019e24c530 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:48.895387 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bcb4cb53-0511-400b-b5a1-c0019e24c530 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03083278661a4cd69237c223badba3f2', 'request_id': 'req-bcb4cb53-0511-400b-b5a1-c0019e24c530'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:48.903260 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bcb4cb53-0511-400b-b5a1-c0019e24c530 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03083278661a4cd69237c223badba3f2', 'request_id': 'req-bcb4cb53-0511-400b-b5a1-c0019e24c530'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:48.917198 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bcb4cb53-0511-400b-b5a1-c0019e24c530 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03083278661a4cd69237c223badba3f2', 'request_id': 'req-bcb4cb53-0511-400b-b5a1-c0019e24c530'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:48.928074 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bcb4cb53-0511-400b-b5a1-c0019e24c530 admin admin] The token cache is being invalidate because project 03083278661a4cd69237c223badba3f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:34:48.928415 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bcb4cb53-0511-400b-b5a1-c0019e24c530 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:48.931178 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bcb4cb53-0511-400b-b5a1-c0019e24c530 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:48.932873 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1033/2071] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:34:48 2022] DELETE /identity/v3/projects/03083278661a4cd69237c223badba3f2 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:48.938800 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca8b5d-e6a3-4cf2-adb2-100f540663b9 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:48.939111 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca8b5d-e6a3-4cf2-adb2-100f540663b9 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:48.939390 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca8b5d-e6a3-4cf2-adb2-100f540663b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:48.982063 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-c8ca8b5d-e6a3-4cf2-adb2-100f540663b9 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:48.983276 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c8ca8b5d-e6a3-4cf2-adb2-100f540663b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:48.989181 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:49.055736 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1039/2072] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:49.065150 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b43d5-d9c7-42c7-9444-be4d5293f994 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:49.065441 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b43d5-d9c7-42c7-9444-be4d5293f994 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:49.065722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b43d5-d9c7-42c7-9444-be4d5293f994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:49.093301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-db9b43d5-d9c7-42c7-9444-be4d5293f994 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:49.094596 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-db9b43d5-d9c7-42c7-9444-be4d5293f994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:49.105502 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1034/2073] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:49.110128 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca8b5d-e6a3-4cf2-adb2-100f540663b9 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:49.120614 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca8b5d-e6a3-4cf2-adb2-100f540663b9 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:49.125396 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b9ce7-9b08-4936-8d4d-a12f684794ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:49.126065 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b9ce7-9b08-4936-8d4d-a12f684794ca admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:49.126350 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b9ce7-9b08-4936-8d4d-a12f684794ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:49.126629 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b9ce7-9b08-4936-8d4d-a12f684794ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:49.127360 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2b9ce7-9b08-4936-8d4d-a12f684794ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:34:49.127666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2b9ce7-9b08-4936-8d4d-a12f684794ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:49.129293 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2b9ce7-9b08-4936-8d4d-a12f684794ca admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:49.141844 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1040/2074] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:34:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:34:49.146292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b43d5-d9c7-42c7-9444-be4d5293f994 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:49.156648 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b43d5-d9c7-42c7-9444-be4d5293f994 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:49.161478 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f28d3-47f7-4b8d-b187-7e8682852c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:49.162142 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f28d3-47f7-4b8d-b187-7e8682852c9b admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:49.162431 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f28d3-47f7-4b8d-b187-7e8682852c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:49.162712 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f28d3-47f7-4b8d-b187-7e8682852c9b admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:49.164216 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f28d3-47f7-4b8d-b187-7e8682852c9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:49.166167 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f28d3-47f7-4b8d-b187-7e8682852c9b admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:49.197620 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1035/2075] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:34:49 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:34:49.202158 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b9ce7-9b08-4936-8d4d-a12f684794ca admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:49.213187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b9ce7-9b08-4936-8d4d-a12f684794ca admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:49.275684 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29090c96-cdcd-480d-a390-51fcda77f91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:49.276392 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29090c96-cdcd-480d-a390-51fcda77f91e admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:49.276680 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29090c96-cdcd-480d-a390-51fcda77f91e admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:49.276961 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29090c96-cdcd-480d-a390-51fcda77f91e admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:49.277653 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29090c96-cdcd-480d-a390-51fcda77f91e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:49.279614 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29090c96-cdcd-480d-a390-51fcda77f91e admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:49.300941 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1041/2076] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:34:49 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:34:49.304736 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f28d3-47f7-4b8d-b187-7e8682852c9b admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:49.315185 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f28d3-47f7-4b8d-b187-7e8682852c9b admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:49.319942 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c863d7-6653-4038-a40e-d74fbcd56380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:49.320641 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c863d7-6653-4038-a40e-d74fbcd56380 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:49.320934 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c863d7-6653-4038-a40e-d74fbcd56380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:49.321219 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c863d7-6653-4038-a40e-d74fbcd56380 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:49.321989 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c863d7-6653-4038-a40e-d74fbcd56380 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:49.323625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c863d7-6653-4038-a40e-d74fbcd56380 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:49.334996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1036/2077] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:34:49.340053 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29090c96-cdcd-480d-a390-51fcda77f91e admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:49.350121 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29090c96-cdcd-480d-a390-51fcda77f91e admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:49.355627 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c9596-2af2-463c-b943-2e27f3080b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:49.356330 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c9596-2af2-463c-b943-2e27f3080b60 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:49.356613 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c9596-2af2-463c-b943-2e27f3080b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:49.356884 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c9596-2af2-463c-b943-2e27f3080b60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:49.357609 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0c9596-2af2-463c-b943-2e27f3080b60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:49.359184 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0c9596-2af2-463c-b943-2e27f3080b60 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:49.370299 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1042/2078] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:34:49.374613 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c863d7-6653-4038-a40e-d74fbcd56380 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:49.385722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c863d7-6653-4038-a40e-d74fbcd56380 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:49.391107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a8266f-55b5-4564-9c34-56d722ebdcde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:49.392012 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8266f-55b5-4564-9c34-56d722ebdcde admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:49.392389 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8266f-55b5-4564-9c34-56d722ebdcde admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:49.392798 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a8266f-55b5-4564-9c34-56d722ebdcde admin admin] PATH_INFO: `/v3/projects/f187cd69b6a64d9781dafe42b1cc6130/users/8ecbd7b93353410fbfbb809cf7f56f25/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:49.409406 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a8266f-55b5-4564-9c34-56d722ebdcde admin admin] RBAC: Authorizing `identity:create_grant(project_id=f187cd69b6a64d9781dafe42b1cc6130, user_id=8ecbd7b93353410fbfbb809cf7f56f25, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:49.412045 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a8266f-55b5-4564-9c34-56d722ebdcde admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:49.425446 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1037/2079] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:34:49 2022] PUT /identity/v3/projects/f187cd69b6a64d9781dafe42b1cc6130/users/8ecbd7b93353410fbfbb809cf7f56f25/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:49.437193 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb3a2c-6dbb-422d-b4b8-feafda5ee7f8 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:49.437712 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb3a2c-6dbb-422d-b4b8-feafda5ee7f8 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:49.437824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb3a2c-6dbb-422d-b4b8-feafda5ee7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:49.465488 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-b5cb3a2c-6dbb-422d-b4b8-feafda5ee7f8 None None] MFA Rules not processed for user `8ecbd7b93353410fbfbb809cf7f56f25`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:49.466691 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b5cb3a2c-6dbb-422d-b4b8-feafda5ee7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:49.530682 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1043/2080] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:34:49.542040 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a8266f-55b5-4564-9c34-56d722ebdcde admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:49.543814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-f4a8266f-55b5-4564-9c34-56d722ebdcde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:49.593057 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:49.608032 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a8266f-55b5-4564-9c34-56d722ebdcde admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:49.612377 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f253916-466d-471d-918a-fbeb5e572906 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:49.613032 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f253916-466d-471d-918a-fbeb5e572906 service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:49.613316 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f253916-466d-471d-918a-fbeb5e572906 service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:49.613582 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f253916-466d-471d-918a-fbeb5e572906 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:49.623494 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f253916-466d-471d-918a-fbeb5e572906 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:49.624975 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f253916-466d-471d-918a-fbeb5e572906 service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:49.635525 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1038/2081] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:34:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:34:52.258431 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc04ccc-10b1-47a3-a35d-4b71a2735ee3 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:52.258938 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc04ccc-10b1-47a3-a35d-4b71a2735ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:52.259016 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc04ccc-10b1-47a3-a35d-4b71a2735ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:52.268320 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:52.297248 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-5dc04ccc-10b1-47a3-a35d-4b71a2735ee3 None None] MFA Rules not processed for user `733f8296ec3d4795a1140c1e1cb6eb95`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:52.298883 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5dc04ccc-10b1-47a3-a35d-4b71a2735ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:52.392438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1044/2082] 10.222.0.16 () {58 vars in 1009 bytes} [Mon May 23 12:34:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:52.429965 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f253916-466d-471d-918a-fbeb5e572906 service cinder] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:52.431460 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-3f253916-466d-471d-918a-fbeb5e572906 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:52.510293 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f253916-466d-471d-918a-fbeb5e572906 service cinder] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:52.516238 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb8f68f-65ee-42d3-81c7-1689a478133f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71fb9a79ac494635804b93d4a0048205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:52.516905 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8f68f-65ee-42d3-81c7-1689a478133f service swift] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:52.517189 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8f68f-65ee-42d3-81c7-1689a478133f service swift] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:52.520459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8f68f-65ee-42d3-81c7-1689a478133f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:52.530339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb8f68f-65ee-42d3-81c7-1689a478133f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:52.532032 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb8f68f-65ee-42d3-81c7-1689a478133f service swift] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:52.544121 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1039/2083] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 12:34:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 12:34:52.973408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc04ccc-10b1-47a3-a35d-4b71a2735ee3 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:53.060977 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc04ccc-10b1-47a3-a35d-4b71a2735ee3 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:53.066168 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0394b1fe-396e-45cf-851f-c9b51114e8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:53.066707 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394b1fe-396e-45cf-851f-c9b51114e8e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:53.066998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394b1fe-396e-45cf-851f-c9b51114e8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:53.067400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0394b1fe-396e-45cf-851f-c9b51114e8e7 admin admin] PATH_INFO: `/v3/users/8ecbd7b93353410fbfbb809cf7f56f25` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:53.068883 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0394b1fe-396e-45cf-851f-c9b51114e8e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ecbd7b93353410fbfbb809cf7f56f25)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:53.071163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0394b1fe-396e-45cf-851f-c9b51114e8e7 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:53.134625 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-0394b1fe-396e-45cf-851f-c9b51114e8e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ecbd7b93353410fbfbb809cf7f56f25'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:53.147018 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-0394b1fe-396e-45cf-851f-c9b51114e8e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ecbd7b93353410fbfbb809cf7f56f25'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:53.156453 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-0394b1fe-396e-45cf-851f-c9b51114e8e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ecbd7b93353410fbfbb809cf7f56f25'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:53.157108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-0394b1fe-396e-45cf-851f-c9b51114e8e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ecbd7b93353410fbfbb809cf7f56f25'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:53.157725 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-0394b1fe-396e-45cf-851f-c9b51114e8e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ecbd7b93353410fbfbb809cf7f56f25'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:53.172546 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1045/2084] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:34:52 2022] DELETE /identity/v3/users/8ecbd7b93353410fbfbb809cf7f56f25 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:34:53.178401 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44639b07-f570-4f72-96b3-3088eecc9bb0 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:53.178401 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44639b07-f570-4f72-96b3-3088eecc9bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:53.178401 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44639b07-f570-4f72-96b3-3088eecc9bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:53.198384 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-44639b07-f570-4f72-96b3-3088eecc9bb0 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:53.199593 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-44639b07-f570-4f72-96b3-3088eecc9bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:53.239212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cfbd03-ea72-4db2-b351-a6a205f16365 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:53.239510 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cfbd03-ea72-4db2-b351-a6a205f16365 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:53.239872 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cfbd03-ea72-4db2-b351-a6a205f16365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:53.259422 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-22cfbd03-ea72-4db2-b351-a6a205f16365 None None] MFA Rules not processed for user `733f8296ec3d4795a1140c1e1cb6eb95`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:53.260717 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-22cfbd03-ea72-4db2-b351-a6a205f16365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:53.276534 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1040/2085] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:53.280044 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:53.288673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44639b07-f570-4f72-96b3-3088eecc9bb0 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:53.292539 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-44639b07-f570-4f72-96b3-3088eecc9bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:53.323006 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1046/2086] 10.222.0.16 () {58 vars in 967 bytes} [Mon May 23 12:34:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:34:53.340737 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cfbd03-ea72-4db2-b351-a6a205f16365 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:53.342860 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-22cfbd03-ea72-4db2-b351-a6a205f16365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:53.354720 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44639b07-f570-4f72-96b3-3088eecc9bb0 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:53.358479 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1394e18-5d57-42b8-a59a-5d29459de7c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:53.359241 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1394e18-5d57-42b8-a59a-5d29459de7c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:53.359602 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1394e18-5d57-42b8-a59a-5d29459de7c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:53.359882 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1394e18-5d57-42b8-a59a-5d29459de7c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:53.371323 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1394e18-5d57-42b8-a59a-5d29459de7c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:53.372866 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1394e18-5d57-42b8-a59a-5d29459de7c0 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:53.384600 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1041/2087] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:34:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:34:53.425192 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cfbd03-ea72-4db2-b351-a6a205f16365 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:53.437517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95defc19-f519-4726-aef5-423db04a809f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71fb9a79ac494635804b93d4a0048205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:53.438312 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95defc19-f519-4726-aef5-423db04a809f service swift] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:53.438843 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95defc19-f519-4726-aef5-423db04a809f service swift] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:53.439028 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95defc19-f519-4726-aef5-423db04a809f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:53.449139 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95defc19-f519-4726-aef5-423db04a809f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:53.451008 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95defc19-f519-4726-aef5-423db04a809f service swift] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:53.460995 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1047/2088] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 12:34:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 23 12:34:53.714414 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1394e18-5d57-42b8-a59a-5d29459de7c0 service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:53.716463 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-e1394e18-5d57-42b8-a59a-5d29459de7c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:53.741067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c36a0-5a2d-414c-a5e3-b32695fdfed5 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:53.742213 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c36a0-5a2d-414c-a5e3-b32695fdfed5 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:53.742668 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c36a0-5a2d-414c-a5e3-b32695fdfed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:53.758139 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1394e18-5d57-42b8-a59a-5d29459de7c0 service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:53.773389 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-b44c36a0-5a2d-414c-a5e3-b32695fdfed5 None None] MFA Rules not processed for user `733f8296ec3d4795a1140c1e1cb6eb95`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:53.775657 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b44c36a0-5a2d-414c-a5e3-b32695fdfed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:53.780666 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43bbf99-bea8-4033-98e8-182a6970ea7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:53.782126 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bbf99-bea8-4033-98e8-182a6970ea7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:53.782606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bbf99-bea8-4033-98e8-182a6970ea7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:53.783076 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43bbf99-bea8-4033-98e8-182a6970ea7f admin admin] PATH_INFO: `/v3/projects/f187cd69b6a64d9781dafe42b1cc6130` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:53.784716 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43bbf99-bea8-4033-98e8-182a6970ea7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f187cd69b6a64d9781dafe42b1cc6130)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:53.787130 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43bbf99-bea8-4033-98e8-182a6970ea7f admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:53.793044 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1048/2089] 10.222.0.16 () {58 vars in 1009 bytes} [Mon May 23 12:34:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:34:53.811406 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c36a0-5a2d-414c-a5e3-b32695fdfed5 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:53.814154 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:53.831204 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c36a0-5a2d-414c-a5e3-b32695fdfed5 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:53.837527 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f7d6b-0d6b-47bf-a5dc-c6b8f4e265d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71fb9a79ac494635804b93d4a0048205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:53.838190 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f7d6b-0d6b-47bf-a5dc-c6b8f4e265d2 service swift] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:53.838543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f7d6b-0d6b-47bf-a5dc-c6b8f4e265d2 service swift] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:53.838807 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f7d6b-0d6b-47bf-a5dc-c6b8f4e265d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:53.849139 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f7d6b-0d6b-47bf-a5dc-c6b8f4e265d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:53.851318 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f7d6b-0d6b-47bf-a5dc-c6b8f4e265d2 service swift] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:53.861378 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1049/2090] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 12:34:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 12:34:53.879460 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f43bbf99-bea8-4033-98e8-182a6970ea7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f187cd69b6a64d9781dafe42b1cc6130', 'request_id': 'req-f43bbf99-bea8-4033-98e8-182a6970ea7f'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:53.902726 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f43bbf99-bea8-4033-98e8-182a6970ea7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f187cd69b6a64d9781dafe42b1cc6130', 'request_id': 'req-f43bbf99-bea8-4033-98e8-182a6970ea7f'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:53.915671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f43bbf99-bea8-4033-98e8-182a6970ea7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f187cd69b6a64d9781dafe42b1cc6130', 'request_id': 'req-f43bbf99-bea8-4033-98e8-182a6970ea7f'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:53.934558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f43bbf99-bea8-4033-98e8-182a6970ea7f admin admin] The token cache is being invalidate because project f187cd69b6a64d9781dafe42b1cc6130 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:34:53.934859 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f43bbf99-bea8-4033-98e8-182a6970ea7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:53.935737 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f43bbf99-bea8-4033-98e8-182a6970ea7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:53.937426 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1042/2091] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:34:53 2022] DELETE /identity/v3/projects/f187cd69b6a64d9781dafe42b1cc6130 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:53.945055 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c281e272-6de9-48e6-9479-eb4eabed6fa3 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:53.945528 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c281e272-6de9-48e6-9479-eb4eabed6fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:53.946110 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c281e272-6de9-48e6-9479-eb4eabed6fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:53.995759 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-c281e272-6de9-48e6-9479-eb4eabed6fa3 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:53.997120 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c281e272-6de9-48e6-9479-eb4eabed6fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:54.091046 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1050/2092] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:54.098740 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6207b-79e2-43e9-b59f-126060081e7d None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:54.099108 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6207b-79e2-43e9-b59f-126060081e7d None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:54.099464 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6207b-79e2-43e9-b59f-126060081e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:54.123970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-6bf6207b-79e2-43e9-b59f-126060081e7d None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:54.125392 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-6bf6207b-79e2-43e9-b59f-126060081e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:54.136820 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1043/2093] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:54.139174 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c281e272-6de9-48e6-9479-eb4eabed6fa3 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:54.150433 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c281e272-6de9-48e6-9479-eb4eabed6fa3 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:54.155231 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bdaa50-3fcc-4b80-978a-5b255c3f825a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:54.155831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bdaa50-3fcc-4b80-978a-5b255c3f825a admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:54.156116 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bdaa50-3fcc-4b80-978a-5b255c3f825a admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:54.156458 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bdaa50-3fcc-4b80-978a-5b255c3f825a admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:54.157187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bdaa50-3fcc-4b80-978a-5b255c3f825a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:34:54.157495 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bdaa50-3fcc-4b80-978a-5b255c3f825a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:54.159102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bdaa50-3fcc-4b80-978a-5b255c3f825a admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:54.174439 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1051/2094] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:34:54.175727 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6207b-79e2-43e9-b59f-126060081e7d None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:54.185856 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6207b-79e2-43e9-b59f-126060081e7d None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:54.191624 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59527ab-6622-4fcf-86fb-bf107c0cd51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:54.193128 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59527ab-6622-4fcf-86fb-bf107c0cd51b admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:54.193495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59527ab-6622-4fcf-86fb-bf107c0cd51b admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:54.193692 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59527ab-6622-4fcf-86fb-bf107c0cd51b admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:54.194960 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59527ab-6622-4fcf-86fb-bf107c0cd51b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:54.197254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59527ab-6622-4fcf-86fb-bf107c0cd51b admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:54.226295 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1044/2095] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:34:54 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:34:54.230454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bdaa50-3fcc-4b80-978a-5b255c3f825a admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:54.242067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bdaa50-3fcc-4b80-978a-5b255c3f825a admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:54.258428 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:54.312266 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb1166-62af-40f3-a534-00be8fa9eb1c None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:54.312661 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb1166-62af-40f3-a534-00be8fa9eb1c None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:54.313010 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb1166-62af-40f3-a534-00be8fa9eb1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:54.313105 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7376ce-4820-49f8-873c-e475ddaf5588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:54.313785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7376ce-4820-49f8-873c-e475ddaf5588 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:54.314075 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7376ce-4820-49f8-873c-e475ddaf5588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:54.314358 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7376ce-4820-49f8-873c-e475ddaf5588 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:54.315049 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7376ce-4820-49f8-873c-e475ddaf5588 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:54.317154 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7376ce-4820-49f8-873c-e475ddaf5588 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:54.332999 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-cffb1166-62af-40f3-a534-00be8fa9eb1c None None] MFA Rules not processed for user `733f8296ec3d4795a1140c1e1cb6eb95`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:54.334296 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-cffb1166-62af-40f3-a534-00be8fa9eb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:54.343363 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1052/2096] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:34:54 2022] POST /identity/v3/users => generated 285 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:34:54.344550 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7376ce-4820-49f8-873c-e475ddaf5588 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:54.355590 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7376ce-4820-49f8-873c-e475ddaf5588 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:54.360301 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c3cbf-c370-4dbc-b203-df01e0c09052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:54.360977 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c3cbf-c370-4dbc-b203-df01e0c09052 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:54.361260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c3cbf-c370-4dbc-b203-df01e0c09052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:54.361541 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c3cbf-c370-4dbc-b203-df01e0c09052 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:54.362272 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c3cbf-c370-4dbc-b203-df01e0c09052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:54.363880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c3cbf-c370-4dbc-b203-df01e0c09052 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:54.378512 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1053/2097] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:34:54.379654 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c3cbf-c370-4dbc-b203-df01e0c09052 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:54.392604 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c3cbf-c370-4dbc-b203-df01e0c09052 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:54.397580 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c539772-8401-450b-846b-40c271674c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:54.398264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c539772-8401-450b-846b-40c271674c02 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:54.398571 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c539772-8401-450b-846b-40c271674c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:54.398834 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c539772-8401-450b-846b-40c271674c02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:54.399562 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c539772-8401-450b-846b-40c271674c02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:54.401217 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c539772-8401-450b-846b-40c271674c02 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:54.418820 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1054/2098] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:34:54.419269 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1045/2099] 10.222.0.16 () {58 vars in 967 bytes} [Mon May 23 12:34:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:34:54.420340 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c539772-8401-450b-846b-40c271674c02 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:54.431745 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb1166-62af-40f3-a534-00be8fa9eb1c None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:54.433245 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-cffb1166-62af-40f3-a534-00be8fa9eb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:54.434610 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c539772-8401-450b-846b-40c271674c02 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:54.440686 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af81e1e-1329-41c4-b31e-b8012e753150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:54.443932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af81e1e-1329-41c4-b31e-b8012e753150 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:54.443932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af81e1e-1329-41c4-b31e-b8012e753150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:54.443932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af81e1e-1329-41c4-b31e-b8012e753150 admin admin] PATH_INFO: `/v3/projects/fb08080f82ff4d73a15c8abd7cb8495f/users/10a62759fc814b1cb53654ccf99f4749/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:54.458434 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af81e1e-1329-41c4-b31e-b8012e753150 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb08080f82ff4d73a15c8abd7cb8495f, user_id=10a62759fc814b1cb53654ccf99f4749, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:54.462464 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af81e1e-1329-41c4-b31e-b8012e753150 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:54.478408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1055/2100] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:34:54 2022] PUT /identity/v3/projects/fb08080f82ff4d73a15c8abd7cb8495f/users/10a62759fc814b1cb53654ccf99f4749/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:34:54.489440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf5071-7dab-41b2-9f49-9c6de4a4af06 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:54.489719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf5071-7dab-41b2-9f49-9c6de4a4af06 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:54.489989 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf5071-7dab-41b2-9f49-9c6de4a4af06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:54.516102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb1166-62af-40f3-a534-00be8fa9eb1c None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:54.518543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-84bf5071-7dab-41b2-9f49-9c6de4a4af06 None None] MFA Rules not processed for user `10a62759fc814b1cb53654ccf99f4749`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:54.519787 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-84bf5071-7dab-41b2-9f49-9c6de4a4af06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:54.535274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:54.583543 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a769841-01a8-4d34-87b0-c68e986c35f1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71fb9a79ac494635804b93d4a0048205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:54.584226 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a769841-01a8-4d34-87b0-c68e986c35f1 service swift] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:54.584521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a769841-01a8-4d34-87b0-c68e986c35f1 service swift] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:54.584803 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a769841-01a8-4d34-87b0-c68e986c35f1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:54.588681 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1056/2101] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:54 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:34:54.601016 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf5071-7dab-41b2-9f49-9c6de4a4af06 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:54.603166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-84bf5071-7dab-41b2-9f49-9c6de4a4af06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:54.645979 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a769841-01a8-4d34-87b0-c68e986c35f1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:54.647647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a769841-01a8-4d34-87b0-c68e986c35f1 service swift] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:54.659553 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1046/2102] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 12:34:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 23 12:34:54.668969 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:54.673172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf5071-7dab-41b2-9f49-9c6de4a4af06 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:54.681684 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffaa07d-4b5b-434e-9472-7fd09fc2b7e8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2483a351f5e943ad8c872a98a4717b0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:54.682358 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaa07d-4b5b-434e-9472-7fd09fc2b7e8 service glance] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:54.682829 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaa07d-4b5b-434e-9472-7fd09fc2b7e8 service glance] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:54.683116 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaa07d-4b5b-434e-9472-7fd09fc2b7e8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:54.694506 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffaa07d-4b5b-434e-9472-7fd09fc2b7e8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:54.696049 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffaa07d-4b5b-434e-9472-7fd09fc2b7e8 service glance] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:54.706874 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1057/2103] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:34:54 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:34:55.028895 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a769841-01a8-4d34-87b0-c68e986c35f1 service swift] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:55.030466 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-3a769841-01a8-4d34-87b0-c68e986c35f1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:55.153853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a769841-01a8-4d34-87b0-c68e986c35f1 service swift] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:55.161267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f0e4b-bda3-43f5-a2fa-413d31fdc433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:55.162168 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f0e4b-bda3-43f5-a2fa-413d31fdc433 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:55.162461 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f0e4b-bda3-43f5-a2fa-413d31fdc433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:55.162743 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f0e4b-bda3-43f5-a2fa-413d31fdc433 admin admin] PATH_INFO: `/v3/users/69bd1f290f3744d4b4d1f873f38efd82` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:55.164996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338f0e4b-bda3-43f5-a2fa-413d31fdc433 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69bd1f290f3744d4b4d1f873f38efd82)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:55.167487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338f0e4b-bda3-43f5-a2fa-413d31fdc433 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:55.282938 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-338f0e4b-bda3-43f5-a2fa-413d31fdc433 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69bd1f290f3744d4b4d1f873f38efd82'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:55.283743 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-338f0e4b-bda3-43f5-a2fa-413d31fdc433 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69bd1f290f3744d4b4d1f873f38efd82'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:55.300225 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-338f0e4b-bda3-43f5-a2fa-413d31fdc433 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69bd1f290f3744d4b4d1f873f38efd82'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:55.323264 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:55.335296 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-338f0e4b-bda3-43f5-a2fa-413d31fdc433 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69bd1f290f3744d4b4d1f873f38efd82'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:55.347391 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-338f0e4b-bda3-43f5-a2fa-413d31fdc433 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69bd1f290f3744d4b4d1f873f38efd82'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:55.362697 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1047/2104] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:34:55 2022] DELETE /identity/v3/users/69bd1f290f3744d4b4d1f873f38efd82 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:55.369691 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd04675-21a7-4aee-ac34-00e318324592 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:55.371245 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd04675-21a7-4aee-ac34-00e318324592 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:55.371683 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd04675-21a7-4aee-ac34-00e318324592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:55.393835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-1fd04675-21a7-4aee-ac34-00e318324592 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:55.395039 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1fd04675-21a7-4aee-ac34-00e318324592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:55.466402 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1058/2105] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:55.476353 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f0e4b-bda3-43f5-a2fa-413d31fdc433 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:55.477906 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-338f0e4b-bda3-43f5-a2fa-413d31fdc433 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:55.558236 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f0e4b-bda3-43f5-a2fa-413d31fdc433 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:55.570084 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da806238-0833-4fee-b8a5-9b13b082a8e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:55.570817 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da806238-0833-4fee-b8a5-9b13b082a8e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:55.571103 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da806238-0833-4fee-b8a5-9b13b082a8e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:55.571531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da806238-0833-4fee-b8a5-9b13b082a8e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:55.583233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da806238-0833-4fee-b8a5-9b13b082a8e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:55.585495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da806238-0833-4fee-b8a5-9b13b082a8e0 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:55.599458 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1048/2106] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:34:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:34:55.936601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd04675-21a7-4aee-ac34-00e318324592 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:55.938081 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1fd04675-21a7-4aee-ac34-00e318324592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:55.956530 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd04675-21a7-4aee-ac34-00e318324592 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:55.963943 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c3ab1-5fea-45b2-a125-2902c99a230e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:55.969091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c3ab1-5fea-45b2-a125-2902c99a230e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:55.969605 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c3ab1-5fea-45b2-a125-2902c99a230e admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:55.970436 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c3ab1-5fea-45b2-a125-2902c99a230e admin admin] PATH_INFO: `/v3/projects/087539c0e4c349c8a06ea352bcc4f3f7` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:55.972912 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29c3ab1-5fea-45b2-a125-2902c99a230e admin admin] RBAC: Authorizing `identity:delete_project(project_id=087539c0e4c349c8a06ea352bcc4f3f7)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:55.977862 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29c3ab1-5fea-45b2-a125-2902c99a230e admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:56.038166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e29c3ab1-5fea-45b2-a125-2902c99a230e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '087539c0e4c349c8a06ea352bcc4f3f7', 'request_id': 'req-e29c3ab1-5fea-45b2-a125-2902c99a230e'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:56.055675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e29c3ab1-5fea-45b2-a125-2902c99a230e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '087539c0e4c349c8a06ea352bcc4f3f7', 'request_id': 'req-e29c3ab1-5fea-45b2-a125-2902c99a230e'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:56.064248 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e29c3ab1-5fea-45b2-a125-2902c99a230e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '087539c0e4c349c8a06ea352bcc4f3f7', 'request_id': 'req-e29c3ab1-5fea-45b2-a125-2902c99a230e'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:56.079161 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e29c3ab1-5fea-45b2-a125-2902c99a230e admin admin] The token cache is being invalidate because project 087539c0e4c349c8a06ea352bcc4f3f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:34:56.079436 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e29c3ab1-5fea-45b2-a125-2902c99a230e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:56.080111 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e29c3ab1-5fea-45b2-a125-2902c99a230e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:56.081793 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1059/2107] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:34:55 2022] DELETE /identity/v3/projects/087539c0e4c349c8a06ea352bcc4f3f7 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:56.086729 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed0bc8-fda9-44e0-8012-b6d0f659c75c None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:56.087273 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed0bc8-fda9-44e0-8012-b6d0f659c75c None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:56.087273 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed0bc8-fda9-44e0-8012-b6d0f659c75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:56.106810 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-37ed0bc8-fda9-44e0-8012-b6d0f659c75c None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:56.108031 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-37ed0bc8-fda9-44e0-8012-b6d0f659c75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:56.181853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1049/2108] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:56.190846 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c56b8c-bfcb-43df-9ee0-bfb677214be8 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:56.191135 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c56b8c-bfcb-43df-9ee0-bfb677214be8 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:56.191678 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c56b8c-bfcb-43df-9ee0-bfb677214be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:56.211568 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-54c56b8c-bfcb-43df-9ee0-bfb677214be8 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:56.212893 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-54c56b8c-bfcb-43df-9ee0-bfb677214be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:56.221753 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1060/2109] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:56.226606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed0bc8-fda9-44e0-8012-b6d0f659c75c None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:56.236679 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed0bc8-fda9-44e0-8012-b6d0f659c75c None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:56.245286 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29767724-2639-4d16-b231-5e48b5da3491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:56.245786 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29767724-2639-4d16-b231-5e48b5da3491 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:56.246068 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29767724-2639-4d16-b231-5e48b5da3491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:56.246447 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29767724-2639-4d16-b231-5e48b5da3491 admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:56.247140 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29767724-2639-4d16-b231-5e48b5da3491 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:34:56.247445 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29767724-2639-4d16-b231-5e48b5da3491 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:56.249120 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29767724-2639-4d16-b231-5e48b5da3491 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:56.271915 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1050/2110] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:34:56.273251 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c56b8c-bfcb-43df-9ee0-bfb677214be8 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:56.289758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c56b8c-bfcb-43df-9ee0-bfb677214be8 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:56.299211 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fbea00-c3e9-4882-abbf-e2d4bc4e8b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:56.299977 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbea00-c3e9-4882-abbf-e2d4bc4e8b8c admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:56.300483 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbea00-c3e9-4882-abbf-e2d4bc4e8b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:56.300824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbea00-c3e9-4882-abbf-e2d4bc4e8b8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:56.301629 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fbea00-c3e9-4882-abbf-e2d4bc4e8b8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:56.303754 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fbea00-c3e9-4882-abbf-e2d4bc4e8b8c admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:56.344114 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1061/2111] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:34:56 2022] POST /identity/v3/projects => generated 383 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:34:56.348252 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29767724-2639-4d16-b231-5e48b5da3491 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:56.363729 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29767724-2639-4d16-b231-5e48b5da3491 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:56.478745 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b2c60-4a1e-4788-9afc-a5c4cc6cf394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:56.479436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b2c60-4a1e-4788-9afc-a5c4cc6cf394 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:56.479791 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b2c60-4a1e-4788-9afc-a5c4cc6cf394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:56.480065 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b2c60-4a1e-4788-9afc-a5c4cc6cf394 admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:56.481114 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b2c60-4a1e-4788-9afc-a5c4cc6cf394 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:56.483287 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b2c60-4a1e-4788-9afc-a5c4cc6cf394 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:56.522023 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1051/2112] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:34:56 2022] POST /identity/v3/users => generated 292 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:34:56.524137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fbea00-c3e9-4882-abbf-e2d4bc4e8b8c admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:56.534876 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fbea00-c3e9-4882-abbf-e2d4bc4e8b8c admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:56.543054 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaef2b8-5a0a-4ce5-90a5-98204e42ed4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:56.543797 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaef2b8-5a0a-4ce5-90a5-98204e42ed4f admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:56.544042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaef2b8-5a0a-4ce5-90a5-98204e42ed4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:56.544408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaef2b8-5a0a-4ce5-90a5-98204e42ed4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:56.545112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaef2b8-5a0a-4ce5-90a5-98204e42ed4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:56.546723 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaef2b8-5a0a-4ce5-90a5-98204e42ed4f admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:56.558084 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1062/2113] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:34:56.562348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b2c60-4a1e-4788-9afc-a5c4cc6cf394 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:56.567696 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:56.574179 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b2c60-4a1e-4788-9afc-a5c4cc6cf394 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:56.581518 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1c717-c610-4fcf-9765-3d8524f7a9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:56.582369 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1c717-c610-4fcf-9765-3d8524f7a9fd admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:56.582862 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1c717-c610-4fcf-9765-3d8524f7a9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:56.583203 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1c717-c610-4fcf-9765-3d8524f7a9fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:56.584114 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1c717-c610-4fcf-9765-3d8524f7a9fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:56.586339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1c717-c610-4fcf-9765-3d8524f7a9fd admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:56.597804 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1052/2114] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:34:56.603167 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaef2b8-5a0a-4ce5-90a5-98204e42ed4f admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:56.613656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaef2b8-5a0a-4ce5-90a5-98204e42ed4f admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:56.619629 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05752d-e50c-4c1a-ad94-6f5290c8eefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:56.620703 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05752d-e50c-4c1a-ad94-6f5290c8eefe admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:56.621163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05752d-e50c-4c1a-ad94-6f5290c8eefe admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:56.621456 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05752d-e50c-4c1a-ad94-6f5290c8eefe admin admin] PATH_INFO: `/v3/projects/e3ee952966f54a1fb20ff093830cf574/users/b963af9875d3429ba79db08ea7f64cef/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:56.639090 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f05752d-e50c-4c1a-ad94-6f5290c8eefe admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3ee952966f54a1fb20ff093830cf574, user_id=b963af9875d3429ba79db08ea7f64cef, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:56.641782 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f05752d-e50c-4c1a-ad94-6f5290c8eefe admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:56.656281 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1063/2115] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:34:56 2022] PUT /identity/v3/projects/e3ee952966f54a1fb20ff093830cf574/users/b963af9875d3429ba79db08ea7f64cef/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:56.667493 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236b830-6d47-4d0d-8bcf-a41690d09e15 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:56.667776 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236b830-6d47-4d0d-8bcf-a41690d09e15 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:56.668060 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236b830-6d47-4d0d-8bcf-a41690d09e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:56.702506 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-4236b830-6d47-4d0d-8bcf-a41690d09e15 None None] MFA Rules not processed for user `b963af9875d3429ba79db08ea7f64cef`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:56.703823 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-4236b830-6d47-4d0d-8bcf-a41690d09e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:56.781262 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1053/2116] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:34:56.793221 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05752d-e50c-4c1a-ad94-6f5290c8eefe admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:56.795071 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-6f05752d-e50c-4c1a-ad94-6f5290c8eefe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:56.815221 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:56.867156 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05752d-e50c-4c1a-ad94-6f5290c8eefe admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:56.872552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eab062-c563-4064-a47b-0caa412dccd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:56.873381 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eab062-c563-4064-a47b-0caa412dccd6 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:56.873771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eab062-c563-4064-a47b-0caa412dccd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:56.874151 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eab062-c563-4064-a47b-0caa412dccd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:56.886823 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eab062-c563-4064-a47b-0caa412dccd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:56.888317 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eab062-c563-4064-a47b-0caa412dccd6 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:56.903911 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1064/2117] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:34:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:34:57.129905 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236b830-6d47-4d0d-8bcf-a41690d09e15 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:57.131441 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-4236b830-6d47-4d0d-8bcf-a41690d09e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:57.214069 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236b830-6d47-4d0d-8bcf-a41690d09e15 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:57.219454 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb461c18-da62-4e22-aeb2-03451de877db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:57.220454 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb461c18-da62-4e22-aeb2-03451de877db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:57.220940 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb461c18-da62-4e22-aeb2-03451de877db admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:57.221313 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb461c18-da62-4e22-aeb2-03451de877db admin admin] PATH_INFO: `/v3/users/c819599f00844589b2957d9587c62d11` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:57.222807 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb461c18-da62-4e22-aeb2-03451de877db admin admin] RBAC: Authorizing `identity:delete_user(user_id=c819599f00844589b2957d9587c62d11)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:57.224875 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb461c18-da62-4e22-aeb2-03451de877db admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:57.285264 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-eb461c18-da62-4e22-aeb2-03451de877db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c819599f00844589b2957d9587c62d11'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:57.285898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-eb461c18-da62-4e22-aeb2-03451de877db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c819599f00844589b2957d9587c62d11'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:57.293696 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-eb461c18-da62-4e22-aeb2-03451de877db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c819599f00844589b2957d9587c62d11'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:57.306424 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-eb461c18-da62-4e22-aeb2-03451de877db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c819599f00844589b2957d9587c62d11'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:57.315052 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-eb461c18-da62-4e22-aeb2-03451de877db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c819599f00844589b2957d9587c62d11'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:57.317591 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:34:57.322797 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1054/2118] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:34:57 2022] DELETE /identity/v3/users/c819599f00844589b2957d9587c62d11 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:34:57.327871 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62789e2e-e3d0-481d-9ea2-3ed717c4e5b4 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:57.328189 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62789e2e-e3d0-481d-9ea2-3ed717c4e5b4 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:57.328560 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62789e2e-e3d0-481d-9ea2-3ed717c4e5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:57.346726 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-62789e2e-e3d0-481d-9ea2-3ed717c4e5b4 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:34:57.347975 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-62789e2e-e3d0-481d-9ea2-3ed717c4e5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:57.413864 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1065/2119] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:34:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:34:57.423606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb461c18-da62-4e22-aeb2-03451de877db admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:57.425269 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-eb461c18-da62-4e22-aeb2-03451de877db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:57.497310 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb461c18-da62-4e22-aeb2-03451de877db admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:57.500999 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295dd517-527f-41b6-bc61-4edba809b4d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:57.501644 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dd517-527f-41b6-bc61-4edba809b4d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:57.501928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dd517-527f-41b6-bc61-4edba809b4d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:57.502207 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dd517-527f-41b6-bc61-4edba809b4d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:57.512928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295dd517-527f-41b6-bc61-4edba809b4d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:57.514730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295dd517-527f-41b6-bc61-4edba809b4d4 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:57.525723 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1055/2120] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:34:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:34:57.759903 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62789e2e-e3d0-481d-9ea2-3ed717c4e5b4 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:34:57.761981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-62789e2e-e3d0-481d-9ea2-3ed717c4e5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:34:57.784932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62789e2e-e3d0-481d-9ea2-3ed717c4e5b4 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:34:57.791258 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dcb1c8-a936-45c5-bd0e-c6e2674174e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:34:57.792922 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dcb1c8-a936-45c5-bd0e-c6e2674174e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:34:57.793741 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dcb1c8-a936-45c5-bd0e-c6e2674174e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:34:57.794037 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dcb1c8-a936-45c5-bd0e-c6e2674174e4 admin admin] PATH_INFO: `/v3/projects/61516a3c24f543d68bdb3724a11f7042` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:34:57.795642 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dcb1c8-a936-45c5-bd0e-c6e2674174e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61516a3c24f543d68bdb3724a11f7042)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:34:57.797606 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dcb1c8-a936-45c5-bd0e-c6e2674174e4 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:34:57.862909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-86dcb1c8-a936-45c5-bd0e-c6e2674174e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61516a3c24f543d68bdb3724a11f7042', 'request_id': 'req-86dcb1c8-a936-45c5-bd0e-c6e2674174e4'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:57.875312 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-86dcb1c8-a936-45c5-bd0e-c6e2674174e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61516a3c24f543d68bdb3724a11f7042', 'request_id': 'req-86dcb1c8-a936-45c5-bd0e-c6e2674174e4'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:57.884079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-86dcb1c8-a936-45c5-bd0e-c6e2674174e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61516a3c24f543d68bdb3724a11f7042', 'request_id': 'req-86dcb1c8-a936-45c5-bd0e-c6e2674174e4'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:57.902457 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-86dcb1c8-a936-45c5-bd0e-c6e2674174e4 admin admin] The token cache is being invalidate because project 61516a3c24f543d68bdb3724a11f7042 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:34:57.903007 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-86dcb1c8-a936-45c5-bd0e-c6e2674174e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:57.903894 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-86dcb1c8-a936-45c5-bd0e-c6e2674174e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:34:57.906110 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1066/2121] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:34:57 2022] DELETE /identity/v3/projects/61516a3c24f543d68bdb3724a11f7042 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:01.610427 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295dd517-527f-41b6-bc61-4edba809b4d4 service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:01.611949 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-295dd517-527f-41b6-bc61-4edba809b4d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:01.641579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295dd517-527f-41b6-bc61-4edba809b4d4 service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:01.716066 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801b3ca-d415-4664-8183-5e339fa89b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:01.717144 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801b3ca-d415-4664-8183-5e339fa89b66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:01.717432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801b3ca-d415-4664-8183-5e339fa89b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:01.717716 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801b3ca-d415-4664-8183-5e339fa89b66 admin admin] PATH_INFO: `/v3/users/b963af9875d3429ba79db08ea7f64cef` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:01.718988 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b801b3ca-d415-4664-8183-5e339fa89b66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b963af9875d3429ba79db08ea7f64cef)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:01.725735 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b801b3ca-d415-4664-8183-5e339fa89b66 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:01.779463 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-b801b3ca-d415-4664-8183-5e339fa89b66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b963af9875d3429ba79db08ea7f64cef'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:01.780133 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-b801b3ca-d415-4664-8183-5e339fa89b66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b963af9875d3429ba79db08ea7f64cef'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:01.787406 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-b801b3ca-d415-4664-8183-5e339fa89b66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b963af9875d3429ba79db08ea7f64cef'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:01.799918 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-b801b3ca-d415-4664-8183-5e339fa89b66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b963af9875d3429ba79db08ea7f64cef'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:01.811263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-b801b3ca-d415-4664-8183-5e339fa89b66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b963af9875d3429ba79db08ea7f64cef'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:01.821315 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1056/2122] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:35:01 2022] DELETE /identity/v3/users/b963af9875d3429ba79db08ea7f64cef => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:01.823335 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccdef72-f207-40b1-a706-67a4251e867c None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:01.823778 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccdef72-f207-40b1-a706-67a4251e867c None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:01.823901 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccdef72-f207-40b1-a706-67a4251e867c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:01.843573 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-9ccdef72-f207-40b1-a706-67a4251e867c None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:01.844824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-9ccdef72-f207-40b1-a706-67a4251e867c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:01.881739 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:01.921157 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1067/2123] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:35:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:35:01.930926 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801b3ca-d415-4664-8183-5e339fa89b66 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:01.932352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-b801b3ca-d415-4664-8183-5e339fa89b66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:01.959125 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:01.995119 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801b3ca-d415-4664-8183-5e339fa89b66 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:01.998760 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05319f65-6635-4eb7-ba6f-3fee86f5b14a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:01.999419 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05319f65-6635-4eb7-ba6f-3fee86f5b14a service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:01.999707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05319f65-6635-4eb7-ba6f-3fee86f5b14a service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:02.000017 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05319f65-6635-4eb7-ba6f-3fee86f5b14a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:02.010599 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05319f65-6635-4eb7-ba6f-3fee86f5b14a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:02.012261 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05319f65-6635-4eb7-ba6f-3fee86f5b14a service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:02.026063 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1057/2124] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:35:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:35:02.313017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccdef72-f207-40b1-a706-67a4251e867c None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:02.314546 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-9ccdef72-f207-40b1-a706-67a4251e867c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:02.329125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccdef72-f207-40b1-a706-67a4251e867c None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:02.335650 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa37010-fb32-4c91-8165-71ee5f210725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:02.336909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa37010-fb32-4c91-8165-71ee5f210725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:02.337369 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa37010-fb32-4c91-8165-71ee5f210725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:02.337811 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa37010-fb32-4c91-8165-71ee5f210725 admin admin] PATH_INFO: `/v3/projects/e3ee952966f54a1fb20ff093830cf574` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:02.339349 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa37010-fb32-4c91-8165-71ee5f210725 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3ee952966f54a1fb20ff093830cf574)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:02.342021 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa37010-fb32-4c91-8165-71ee5f210725 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:02.411569 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9fa37010-fb32-4c91-8165-71ee5f210725 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3ee952966f54a1fb20ff093830cf574', 'request_id': 'req-9fa37010-fb32-4c91-8165-71ee5f210725'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:02.420375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9fa37010-fb32-4c91-8165-71ee5f210725 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3ee952966f54a1fb20ff093830cf574', 'request_id': 'req-9fa37010-fb32-4c91-8165-71ee5f210725'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:02.427827 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9fa37010-fb32-4c91-8165-71ee5f210725 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3ee952966f54a1fb20ff093830cf574', 'request_id': 'req-9fa37010-fb32-4c91-8165-71ee5f210725'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:02.442386 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9fa37010-fb32-4c91-8165-71ee5f210725 admin admin] The token cache is being invalidate because project e3ee952966f54a1fb20ff093830cf574 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:35:02.442923 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9fa37010-fb32-4c91-8165-71ee5f210725 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:02.443628 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9fa37010-fb32-4c91-8165-71ee5f210725 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:02.445444 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1068/2125] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:35:02 2022] DELETE /identity/v3/projects/e3ee952966f54a1fb20ff093830cf574 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:12.018033 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=16 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:12.438905 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=37,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:18.138145 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05319f65-6635-4eb7-ba6f-3fee86f5b14a service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:18.139633 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-05319f65-6635-4eb7-ba6f-3fee86f5b14a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:18.155516 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05319f65-6635-4eb7-ba6f-3fee86f5b14a service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:18.240133 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da361bac-73a3-42d3-9910-ab2a1d968256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:18.241555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da361bac-73a3-42d3-9910-ab2a1d968256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:18.241990 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da361bac-73a3-42d3-9910-ab2a1d968256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:18.242431 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da361bac-73a3-42d3-9910-ab2a1d968256 admin admin] PATH_INFO: `/v3/users/e981c1fed59d43c48981ad7ddcafb234` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:18.244308 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da361bac-73a3-42d3-9910-ab2a1d968256 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e981c1fed59d43c48981ad7ddcafb234)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:18.247409 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da361bac-73a3-42d3-9910-ab2a1d968256 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:18.308905 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-da361bac-73a3-42d3-9910-ab2a1d968256 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e981c1fed59d43c48981ad7ddcafb234'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:18.309590 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-da361bac-73a3-42d3-9910-ab2a1d968256 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e981c1fed59d43c48981ad7ddcafb234'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:18.317235 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-da361bac-73a3-42d3-9910-ab2a1d968256 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e981c1fed59d43c48981ad7ddcafb234'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:18.333869 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-da361bac-73a3-42d3-9910-ab2a1d968256 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e981c1fed59d43c48981ad7ddcafb234'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:18.343797 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-da361bac-73a3-42d3-9910-ab2a1d968256 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e981c1fed59d43c48981ad7ddcafb234'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:18.350436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1058/2126] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:35:18 2022] DELETE /identity/v3/users/e981c1fed59d43c48981ad7ddcafb234 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:18.356136 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef5aca-9365-4ed5-aab7-674528f0868e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:18.356643 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef5aca-9365-4ed5-aab7-674528f0868e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:18.357142 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef5aca-9365-4ed5-aab7-674528f0868e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:18.377457 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-41ef5aca-9365-4ed5-aab7-674528f0868e None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:18.378797 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-41ef5aca-9365-4ed5-aab7-674528f0868e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:18.453150 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1069/2127] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:35:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:35:18.465941 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da361bac-73a3-42d3-9910-ab2a1d968256 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:18.467928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-da361bac-73a3-42d3-9910-ab2a1d968256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:18.553291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da361bac-73a3-42d3-9910-ab2a1d968256 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:18.567988 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f3db3-ca39-4225-9230-719b2aab94af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:18.569770 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f3db3-ca39-4225-9230-719b2aab94af service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:18.569770 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f3db3-ca39-4225-9230-719b2aab94af service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:18.580988 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f3db3-ca39-4225-9230-719b2aab94af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:18.605856 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f3db3-ca39-4225-9230-719b2aab94af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:18.608454 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f3db3-ca39-4225-9230-719b2aab94af service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:18.637430 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1059/2128] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:35:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:35:19.122155 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef5aca-9365-4ed5-aab7-674528f0868e None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:19.123708 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-41ef5aca-9365-4ed5-aab7-674528f0868e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:19.136503 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef5aca-9365-4ed5-aab7-674528f0868e None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:19.142508 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee1039-11c8-4bdc-a3c1-91c3a2148fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:19.143190 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee1039-11c8-4bdc-a3c1-91c3a2148fdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:19.143477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee1039-11c8-4bdc-a3c1-91c3a2148fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:19.143911 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee1039-11c8-4bdc-a3c1-91c3a2148fdd admin admin] PATH_INFO: `/v3/projects/c5c98a8a099f4628acf38de147718198` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:19.145276 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee1039-11c8-4bdc-a3c1-91c3a2148fdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5c98a8a099f4628acf38de147718198)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:19.151891 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee1039-11c8-4bdc-a3c1-91c3a2148fdd admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:19.203975 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-97ee1039-11c8-4bdc-a3c1-91c3a2148fdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5c98a8a099f4628acf38de147718198', 'request_id': 'req-97ee1039-11c8-4bdc-a3c1-91c3a2148fdd'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:19.209874 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-97ee1039-11c8-4bdc-a3c1-91c3a2148fdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5c98a8a099f4628acf38de147718198', 'request_id': 'req-97ee1039-11c8-4bdc-a3c1-91c3a2148fdd'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:19.216487 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-97ee1039-11c8-4bdc-a3c1-91c3a2148fdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5c98a8a099f4628acf38de147718198', 'request_id': 'req-97ee1039-11c8-4bdc-a3c1-91c3a2148fdd'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:19.229853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-97ee1039-11c8-4bdc-a3c1-91c3a2148fdd admin admin] The token cache is being invalidate because project c5c98a8a099f4628acf38de147718198 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:35:19.230294 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-97ee1039-11c8-4bdc-a3c1-91c3a2148fdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:19.231011 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-97ee1039-11c8-4bdc-a3c1-91c3a2148fdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:19.232617 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1070/2129] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:35:19 2022] DELETE /identity/v3/projects/c5c98a8a099f4628acf38de147718198 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:19.236592 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f3db3-ca39-4225-9230-719b2aab94af service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:19.248689 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f3db3-ca39-4225-9230-719b2aab94af service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:19.271475 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:19.323343 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59933ea4-6237-4cdd-a6c9-e1d038917e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:19.324259 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59933ea4-6237-4cdd-a6c9-e1d038917e3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:19.324548 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59933ea4-6237-4cdd-a6c9-e1d038917e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:19.324827 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59933ea4-6237-4cdd-a6c9-e1d038917e3b admin admin] PATH_INFO: `/v3/users/cc557049aadf44b6b250c8e08ed77d31` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:19.326285 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59933ea4-6237-4cdd-a6c9-e1d038917e3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc557049aadf44b6b250c8e08ed77d31)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:19.328294 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59933ea4-6237-4cdd-a6c9-e1d038917e3b admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:19.380346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-59933ea4-6237-4cdd-a6c9-e1d038917e3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc557049aadf44b6b250c8e08ed77d31'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:19.380913 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-59933ea4-6237-4cdd-a6c9-e1d038917e3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc557049aadf44b6b250c8e08ed77d31'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:19.388409 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-59933ea4-6237-4cdd-a6c9-e1d038917e3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc557049aadf44b6b250c8e08ed77d31'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:19.400744 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-59933ea4-6237-4cdd-a6c9-e1d038917e3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc557049aadf44b6b250c8e08ed77d31'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:19.408412 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-59933ea4-6237-4cdd-a6c9-e1d038917e3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc557049aadf44b6b250c8e08ed77d31'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:19.415684 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1060/2130] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:35:19 2022] DELETE /identity/v3/users/cc557049aadf44b6b250c8e08ed77d31 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:19.707627 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee1039-11c8-4bdc-a3c1-91c3a2148fdd admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:19.709085 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-97ee1039-11c8-4bdc-a3c1-91c3a2148fdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:19.727894 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee1039-11c8-4bdc-a3c1-91c3a2148fdd admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:19.793524 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea64a73-fee8-4dec-b93c-f10d5303844b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:19.794405 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea64a73-fee8-4dec-b93c-f10d5303844b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:19.794693 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea64a73-fee8-4dec-b93c-f10d5303844b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:19.794980 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea64a73-fee8-4dec-b93c-f10d5303844b admin admin] PATH_INFO: `/v3/projects/2f9687dbe3b34f259c044297eaf3ce04` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:19.796198 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea64a73-fee8-4dec-b93c-f10d5303844b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f9687dbe3b34f259c044297eaf3ce04)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:19.798141 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea64a73-fee8-4dec-b93c-f10d5303844b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:19.832125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:19.852626 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6ea64a73-fee8-4dec-b93c-f10d5303844b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f9687dbe3b34f259c044297eaf3ce04', 'request_id': 'req-6ea64a73-fee8-4dec-b93c-f10d5303844b'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:19.858064 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6ea64a73-fee8-4dec-b93c-f10d5303844b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f9687dbe3b34f259c044297eaf3ce04', 'request_id': 'req-6ea64a73-fee8-4dec-b93c-f10d5303844b'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:19.864643 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6ea64a73-fee8-4dec-b93c-f10d5303844b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f9687dbe3b34f259c044297eaf3ce04', 'request_id': 'req-6ea64a73-fee8-4dec-b93c-f10d5303844b'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:19.880032 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6ea64a73-fee8-4dec-b93c-f10d5303844b admin admin] The token cache is being invalidate because project 2f9687dbe3b34f259c044297eaf3ce04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:35:19.880371 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6ea64a73-fee8-4dec-b93c-f10d5303844b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:19.881617 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6ea64a73-fee8-4dec-b93c-f10d5303844b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:19.884050 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1071/2131] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:35:19 2022] DELETE /identity/v3/projects/2f9687dbe3b34f259c044297eaf3ce04 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:19.888858 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d7e01-349e-4565-8783-79ac62748d39 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:19.889263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d7e01-349e-4565-8783-79ac62748d39 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:19.889645 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d7e01-349e-4565-8783-79ac62748d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:19.923013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-c18d7e01-349e-4565-8783-79ac62748d39 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:19.924497 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-c18d7e01-349e-4565-8783-79ac62748d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:20.028761 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1061/2132] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:35:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:35:20.036759 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a4da9-bedd-4a39-bb81-00d4eea7394e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:20.037051 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a4da9-bedd-4a39-bb81-00d4eea7394e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:20.037337 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a4da9-bedd-4a39-bb81-00d4eea7394e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:20.060186 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-cf9a4da9-bedd-4a39-bb81-00d4eea7394e None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:20.061815 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-cf9a4da9-bedd-4a39-bb81-00d4eea7394e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:20.073372 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1072/2133] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:35:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:35:20.078913 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d7e01-349e-4565-8783-79ac62748d39 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:20.094189 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d7e01-349e-4565-8783-79ac62748d39 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:20.099556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbacad99-ae9c-4217-b84e-af50a7dac7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:20.100263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacad99-ae9c-4217-b84e-af50a7dac7b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:20.100552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacad99-ae9c-4217-b84e-af50a7dac7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:20.100959 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacad99-ae9c-4217-b84e-af50a7dac7b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:20.102253 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbacad99-ae9c-4217-b84e-af50a7dac7b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:35:20.102753 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbacad99-ae9c-4217-b84e-af50a7dac7b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:20.104434 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbacad99-ae9c-4217-b84e-af50a7dac7b4 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:20.119491 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1062/2134] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:35:20.123838 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a4da9-bedd-4a39-bb81-00d4eea7394e None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:20.134255 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a4da9-bedd-4a39-bb81-00d4eea7394e None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:20.139572 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00de39-04c1-4091-8716-084b77fcec9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:20.140338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00de39-04c1-4091-8716-084b77fcec9b admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:20.140623 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00de39-04c1-4091-8716-084b77fcec9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:20.140899 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00de39-04c1-4091-8716-084b77fcec9b admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:20.141928 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b00de39-04c1-4091-8716-084b77fcec9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:20.143856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b00de39-04c1-4091-8716-084b77fcec9b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:20.178095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1073/2135] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:35:20 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:35:20.183502 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbacad99-ae9c-4217-b84e-af50a7dac7b4 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:20.197512 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbacad99-ae9c-4217-b84e-af50a7dac7b4 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:20.234929 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:20.280040 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c318a-e9da-4ae7-9e9e-7ccff660a01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:20.280768 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c318a-e9da-4ae7-9e9e-7ccff660a01a admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:20.281050 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c318a-e9da-4ae7-9e9e-7ccff660a01a admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:20.281317 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c318a-e9da-4ae7-9e9e-7ccff660a01a admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:20.282182 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c318a-e9da-4ae7-9e9e-7ccff660a01a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:20.284284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c318a-e9da-4ae7-9e9e-7ccff660a01a admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:20.319612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1063/2136] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:35:20 2022] POST /identity/v3/users => generated 290 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:35:20.325116 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00de39-04c1-4091-8716-084b77fcec9b admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:20.341948 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00de39-04c1-4091-8716-084b77fcec9b admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:20.354063 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083782bd-6259-4e2a-9132-d82145bb8204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:20.354817 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083782bd-6259-4e2a-9132-d82145bb8204 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:20.355110 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083782bd-6259-4e2a-9132-d82145bb8204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:20.355477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083782bd-6259-4e2a-9132-d82145bb8204 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:20.356285 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083782bd-6259-4e2a-9132-d82145bb8204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:20.357934 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083782bd-6259-4e2a-9132-d82145bb8204 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:20.373435 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1074/2137] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:35:20.374674 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c318a-e9da-4ae7-9e9e-7ccff660a01a admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:20.387580 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c318a-e9da-4ae7-9e9e-7ccff660a01a admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:20.392961 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f76b9-8c05-4063-883d-45bcab0d6a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:20.393644 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f76b9-8c05-4063-883d-45bcab0d6a10 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:20.393930 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f76b9-8c05-4063-883d-45bcab0d6a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:20.394212 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f76b9-8c05-4063-883d-45bcab0d6a10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:20.394976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f76b9-8c05-4063-883d-45bcab0d6a10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:20.396609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f76b9-8c05-4063-883d-45bcab0d6a10 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:20.407945 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1064/2138] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:35:20.412333 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083782bd-6259-4e2a-9132-d82145bb8204 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:20.423496 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083782bd-6259-4e2a-9132-d82145bb8204 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:20.428517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89dc813-24a6-4622-87a4-4e11294e1adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:20.429656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89dc813-24a6-4622-87a4-4e11294e1adb admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:20.430601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89dc813-24a6-4622-87a4-4e11294e1adb admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:20.430969 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89dc813-24a6-4622-87a4-4e11294e1adb admin admin] PATH_INFO: `/v3/projects/d6e616d301d6472f8c8b6cec5b0998da/users/0b73d0d2e36a4dab8055ae05afd5dfe2/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:20.447394 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89dc813-24a6-4622-87a4-4e11294e1adb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6e616d301d6472f8c8b6cec5b0998da, user_id=0b73d0d2e36a4dab8055ae05afd5dfe2, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:20.450113 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89dc813-24a6-4622-87a4-4e11294e1adb admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:20.465995 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1075/2139] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:35:20 2022] PUT /identity/v3/projects/d6e616d301d6472f8c8b6cec5b0998da/users/0b73d0d2e36a4dab8055ae05afd5dfe2/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:20.477851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec17f4d-dc1a-4aff-8eda-ad7a7584692f None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:20.478140 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec17f4d-dc1a-4aff-8eda-ad7a7584692f None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:20.478419 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec17f4d-dc1a-4aff-8eda-ad7a7584692f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:20.507735 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-fec17f4d-dc1a-4aff-8eda-ad7a7584692f None None] MFA Rules not processed for user `0b73d0d2e36a4dab8055ae05afd5dfe2`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:20.509126 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-fec17f4d-dc1a-4aff-8eda-ad7a7584692f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:20.578589 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1065/2140] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:35:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:35:20.583322 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89dc813-24a6-4622-87a4-4e11294e1adb admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:20.651499 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89dc813-24a6-4622-87a4-4e11294e1adb admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:20.656370 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3571425-fbb0-4c91-83ed-4f849e8ae876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:20.657043 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3571425-fbb0-4c91-83ed-4f849e8ae876 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:20.657327 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3571425-fbb0-4c91-83ed-4f849e8ae876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:20.657606 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3571425-fbb0-4c91-83ed-4f849e8ae876 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:20.658332 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3571425-fbb0-4c91-83ed-4f849e8ae876 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:20.660315 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3571425-fbb0-4c91-83ed-4f849e8ae876 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:20.692885 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1076/2141] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:35:20 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:35:20.694062 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec17f4d-dc1a-4aff-8eda-ad7a7584692f None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:20.705634 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec17f4d-dc1a-4aff-8eda-ad7a7584692f None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:20.777979 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5408b3c-80cc-4893-b6a9-780ec91b5e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:20.778675 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5408b3c-80cc-4893-b6a9-780ec91b5e54 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:20.778978 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5408b3c-80cc-4893-b6a9-780ec91b5e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:20.779876 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5408b3c-80cc-4893-b6a9-780ec91b5e54 admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:20.780632 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5408b3c-80cc-4893-b6a9-780ec91b5e54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:20.782625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5408b3c-80cc-4893-b6a9-780ec91b5e54 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:20.804353 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1066/2142] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:35:20 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:35:20.808763 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3571425-fbb0-4c91-83ed-4f849e8ae876 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:20.819412 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3571425-fbb0-4c91-83ed-4f849e8ae876 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:20.824901 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e737e-591f-4d44-ba29-0669d6b0709c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:20.825586 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e737e-591f-4d44-ba29-0669d6b0709c admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:20.826183 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e737e-591f-4d44-ba29-0669d6b0709c admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:20.826411 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e737e-591f-4d44-ba29-0669d6b0709c admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:20.827205 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e737e-591f-4d44-ba29-0669d6b0709c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:20.828905 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e737e-591f-4d44-ba29-0669d6b0709c admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:20.841376 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1077/2143] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:35:20.845847 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5408b3c-80cc-4893-b6a9-780ec91b5e54 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:20.857067 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5408b3c-80cc-4893-b6a9-780ec91b5e54 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:20.861974 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c8bc3-4483-414a-8d66-93b24262ca79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:20.862890 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c8bc3-4483-414a-8d66-93b24262ca79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:20.863156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c8bc3-4483-414a-8d66-93b24262ca79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:20.863431 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c8bc3-4483-414a-8d66-93b24262ca79 admin admin] PATH_INFO: `/v3/projects/2aeb1da0ee5d47b5a40efd982d849130/users/172f8711bd2b4ccabe67e1e7b0e9a171/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:20.880400 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c8bc3-4483-414a-8d66-93b24262ca79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aeb1da0ee5d47b5a40efd982d849130, user_id=172f8711bd2b4ccabe67e1e7b0e9a171, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:20.883768 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c8bc3-4483-414a-8d66-93b24262ca79 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:20.898261 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1067/2144] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:35:20 2022] PUT /identity/v3/projects/2aeb1da0ee5d47b5a40efd982d849130/users/172f8711bd2b4ccabe67e1e7b0e9a171/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:20.903113 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e737e-591f-4d44-ba29-0669d6b0709c admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:20.913783 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e737e-591f-4d44-ba29-0669d6b0709c admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:20.940655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:20.986565 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2245e03-8362-4ac0-914b-1cd89e5dd161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:20.987252 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2245e03-8362-4ac0-914b-1cd89e5dd161 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:20.988931 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2245e03-8362-4ac0-914b-1cd89e5dd161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:20.988931 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2245e03-8362-4ac0-914b-1cd89e5dd161 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:20.988931 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2245e03-8362-4ac0-914b-1cd89e5dd161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:20.990091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2245e03-8362-4ac0-914b-1cd89e5dd161 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:21.000819 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1078/2145] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:35:21.005177 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c8bc3-4483-414a-8d66-93b24262ca79 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:21.012399 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:21.020040 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c8bc3-4483-414a-8d66-93b24262ca79 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:21.026291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ec524-a6f6-4ae7-8928-c38e20264f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:21.027661 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ec524-a6f6-4ae7-8928-c38e20264f3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:21.027954 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ec524-a6f6-4ae7-8928-c38e20264f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:21.028257 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ec524-a6f6-4ae7-8928-c38e20264f3d admin admin] PATH_INFO: `/v3/projects/2aeb1da0ee5d47b5a40efd982d849130/users/172f8711bd2b4ccabe67e1e7b0e9a171/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:21.030268 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ec524-a6f6-4ae7-8928-c38e20264f3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aeb1da0ee5d47b5a40efd982d849130, user_id=172f8711bd2b4ccabe67e1e7b0e9a171, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:21.033048 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ec524-a6f6-4ae7-8928-c38e20264f3d admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:21.044835 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1068/2146] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:35:21 2022] PUT /identity/v3/projects/2aeb1da0ee5d47b5a40efd982d849130/users/172f8711bd2b4ccabe67e1e7b0e9a171/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:21.056376 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7b192-e297-4ccd-9dde-6e305ba46b63 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:21.056673 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7b192-e297-4ccd-9dde-6e305ba46b63 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:21.056954 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e7b192-e297-4ccd-9dde-6e305ba46b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:21.085837 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-88e7b192-e297-4ccd-9dde-6e305ba46b63 None None] MFA Rules not processed for user `172f8711bd2b4ccabe67e1e7b0e9a171`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:21.087645 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-88e7b192-e297-4ccd-9dde-6e305ba46b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:21.168757 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1079/2147] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:35:21 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:35:21.181417 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ec524-a6f6-4ae7-8928-c38e20264f3d admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:21.186293 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-190ec524-a6f6-4ae7-8928-c38e20264f3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:21.282393 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ec524-a6f6-4ae7-8928-c38e20264f3d admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:21.286652 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2daa92-398c-4913-a856-970c689de661 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:21.289541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2daa92-398c-4913-a856-970c689de661 service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:21.289541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2daa92-398c-4913-a856-970c689de661 service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:21.289541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2daa92-398c-4913-a856-970c689de661 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:21.300042 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2daa92-398c-4913-a856-970c689de661 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:21.301604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2daa92-398c-4913-a856-970c689de661 service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:21.312359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1069/2148] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:35:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:35:21.402305 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e7b192-e297-4ccd-9dde-6e305ba46b63 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:21.404130 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-88e7b192-e297-4ccd-9dde-6e305ba46b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:21.426853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e7b192-e297-4ccd-9dde-6e305ba46b63 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:21.431426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e53ad3-1270-411c-9f15-58a79297d0df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:21.432087 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e53ad3-1270-411c-9f15-58a79297d0df service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:21.432441 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e53ad3-1270-411c-9f15-58a79297d0df service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:21.432724 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e53ad3-1270-411c-9f15-58a79297d0df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:21.494856 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e53ad3-1270-411c-9f15-58a79297d0df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:21.496355 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e53ad3-1270-411c-9f15-58a79297d0df service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:21.505777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1080/2149] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:35:21 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:35:31.307885 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=39,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:31.485340 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2daa92-398c-4913-a856-970c689de661 service cinder] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:31.501361 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=80 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:31.560520 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2daa92-398c-4913-a856-970c689de661 service cinder] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:31.565409 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326178ae-9ba2-419d-a90e-e1f3c8126b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:31.566510 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326178ae-9ba2-419d-a90e-e1f3c8126b93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:31.566676 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326178ae-9ba2-419d-a90e-e1f3c8126b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:31.567340 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326178ae-9ba2-419d-a90e-e1f3c8126b93 admin admin] PATH_INFO: `/v3/users/0b73d0d2e36a4dab8055ae05afd5dfe2` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:31.568355 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326178ae-9ba2-419d-a90e-e1f3c8126b93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b73d0d2e36a4dab8055ae05afd5dfe2)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:31.571747 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326178ae-9ba2-419d-a90e-e1f3c8126b93 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:31.606316 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfcf56b-ebed-4a8e-a04e-edc62c8a0ed6 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:31.606689 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfcf56b-ebed-4a8e-a04e-edc62c8a0ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:31.606934 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfcf56b-ebed-4a8e-a04e-edc62c8a0ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:31.627644 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-3bfcf56b-ebed-4a8e-a04e-edc62c8a0ed6 None None] MFA Rules not processed for user `ef222fe025534a9e942f8c5c4ed1e3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:31.628812 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3bfcf56b-ebed-4a8e-a04e-edc62c8a0ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:31.633285 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-326178ae-9ba2-419d-a90e-e1f3c8126b93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b73d0d2e36a4dab8055ae05afd5dfe2'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:31.633966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-326178ae-9ba2-419d-a90e-e1f3c8126b93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b73d0d2e36a4dab8055ae05afd5dfe2'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:31.641339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-326178ae-9ba2-419d-a90e-e1f3c8126b93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b73d0d2e36a4dab8055ae05afd5dfe2'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:31.655286 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-326178ae-9ba2-419d-a90e-e1f3c8126b93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b73d0d2e36a4dab8055ae05afd5dfe2'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:31.663543 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-326178ae-9ba2-419d-a90e-e1f3c8126b93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b73d0d2e36a4dab8055ae05afd5dfe2'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:31.670555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1070/2150] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:35:31 2022] DELETE /identity/v3/users/0b73d0d2e36a4dab8055ae05afd5dfe2 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:31.675071 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11acc3-df22-4e28-a81e-8b1c90a0309e None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:31.675362 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11acc3-df22-4e28-a81e-8b1c90a0309e None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:31.675650 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11acc3-df22-4e28-a81e-8b1c90a0309e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:31.689273 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1081/2151] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:35:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:35:31.701283 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-fc11acc3-df22-4e28-a81e-8b1c90a0309e None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:31.703043 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-fc11acc3-df22-4e28-a81e-8b1c90a0309e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:31.770858 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1071/2152] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:35:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:35:31.782172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfcf56b-ebed-4a8e-a04e-edc62c8a0ed6 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:31.783670 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3bfcf56b-ebed-4a8e-a04e-edc62c8a0ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:31.848937 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfcf56b-ebed-4a8e-a04e-edc62c8a0ed6 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:31.908514 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7fc9b-62fa-4ef8-a589-f9a10d5937e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:31.909184 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7fc9b-62fa-4ef8-a589-f9a10d5937e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:31.910275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7fc9b-62fa-4ef8-a589-f9a10d5937e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:31.910573 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7fc9b-62fa-4ef8-a589-f9a10d5937e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:31.933557 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7fc9b-62fa-4ef8-a589-f9a10d5937e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:31.935103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7fc9b-62fa-4ef8-a589-f9a10d5937e7 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:31.948902 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1082/2153] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:35:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:35:32.592676 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc11acc3-df22-4e28-a81e-8b1c90a0309e None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:32.595191 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-fc11acc3-df22-4e28-a81e-8b1c90a0309e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:32.623624 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc11acc3-df22-4e28-a81e-8b1c90a0309e None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:32.628232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52feef64-ee01-49f7-b6f6-d242eafeb0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:32.629298 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52feef64-ee01-49f7-b6f6-d242eafeb0d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:32.629597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52feef64-ee01-49f7-b6f6-d242eafeb0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:32.629871 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52feef64-ee01-49f7-b6f6-d242eafeb0d0 admin admin] PATH_INFO: `/v3/projects/d6e616d301d6472f8c8b6cec5b0998da` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:32.631081 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52feef64-ee01-49f7-b6f6-d242eafeb0d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6e616d301d6472f8c8b6cec5b0998da)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:32.634854 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52feef64-ee01-49f7-b6f6-d242eafeb0d0 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:32.669830 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:32.725593 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-52feef64-ee01-49f7-b6f6-d242eafeb0d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6e616d301d6472f8c8b6cec5b0998da', 'request_id': 'req-52feef64-ee01-49f7-b6f6-d242eafeb0d0'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:32.736810 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-52feef64-ee01-49f7-b6f6-d242eafeb0d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6e616d301d6472f8c8b6cec5b0998da', 'request_id': 'req-52feef64-ee01-49f7-b6f6-d242eafeb0d0'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:32.756196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-52feef64-ee01-49f7-b6f6-d242eafeb0d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6e616d301d6472f8c8b6cec5b0998da', 'request_id': 'req-52feef64-ee01-49f7-b6f6-d242eafeb0d0'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:32.774333 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-52feef64-ee01-49f7-b6f6-d242eafeb0d0 admin admin] The token cache is being invalidate because project d6e616d301d6472f8c8b6cec5b0998da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:35:32.774705 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-52feef64-ee01-49f7-b6f6-d242eafeb0d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:32.775432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-52feef64-ee01-49f7-b6f6-d242eafeb0d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:32.777121 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1072/2154] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:35:32 2022] DELETE /identity/v3/projects/d6e616d301d6472f8c8b6cec5b0998da => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:32.781115 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7fc9b-62fa-4ef8-a589-f9a10d5937e7 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:32.791999 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7fc9b-62fa-4ef8-a589-f9a10d5937e7 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:32.866638 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42edbb5f-5c40-4638-b5fe-9bb371335586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:32.868997 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42edbb5f-5c40-4638-b5fe-9bb371335586 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:32.869258 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42edbb5f-5c40-4638-b5fe-9bb371335586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:32.869536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42edbb5f-5c40-4638-b5fe-9bb371335586 admin admin] PATH_INFO: `/v3/users/172f8711bd2b4ccabe67e1e7b0e9a171` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:32.870913 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42edbb5f-5c40-4638-b5fe-9bb371335586 admin admin] RBAC: Authorizing `identity:delete_user(user_id=172f8711bd2b4ccabe67e1e7b0e9a171)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:32.873408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42edbb5f-5c40-4638-b5fe-9bb371335586 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:32.878846 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:32.933249 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-42edbb5f-5c40-4638-b5fe-9bb371335586 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '172f8711bd2b4ccabe67e1e7b0e9a171'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:32.949424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-42edbb5f-5c40-4638-b5fe-9bb371335586 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '172f8711bd2b4ccabe67e1e7b0e9a171'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:32.958141 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-42edbb5f-5c40-4638-b5fe-9bb371335586 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '172f8711bd2b4ccabe67e1e7b0e9a171'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:32.959544 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-42edbb5f-5c40-4638-b5fe-9bb371335586 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '172f8711bd2b4ccabe67e1e7b0e9a171'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:32.960324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-42edbb5f-5c40-4638-b5fe-9bb371335586 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '172f8711bd2b4ccabe67e1e7b0e9a171'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:32.974245 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1083/2155] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:35:32 2022] DELETE /identity/v3/users/172f8711bd2b4ccabe67e1e7b0e9a171 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:33.337211 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52feef64-ee01-49f7-b6f6-d242eafeb0d0 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:33.338654 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-52feef64-ee01-49f7-b6f6-d242eafeb0d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:33.355849 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52feef64-ee01-49f7-b6f6-d242eafeb0d0 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:33.442520 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb5ce6-c6de-433e-bdbf-51d8ac9e6210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:33.443639 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb5ce6-c6de-433e-bdbf-51d8ac9e6210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:33.444127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb5ce6-c6de-433e-bdbf-51d8ac9e6210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:33.444576 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb5ce6-c6de-433e-bdbf-51d8ac9e6210 admin admin] PATH_INFO: `/v3/projects/2aeb1da0ee5d47b5a40efd982d849130` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:33.446758 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdb5ce6-c6de-433e-bdbf-51d8ac9e6210 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aeb1da0ee5d47b5a40efd982d849130)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:33.450620 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdb5ce6-c6de-433e-bdbf-51d8ac9e6210 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:33.511858 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bcdb5ce6-c6de-433e-bdbf-51d8ac9e6210 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aeb1da0ee5d47b5a40efd982d849130', 'request_id': 'req-bcdb5ce6-c6de-433e-bdbf-51d8ac9e6210'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:33.519990 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bcdb5ce6-c6de-433e-bdbf-51d8ac9e6210 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aeb1da0ee5d47b5a40efd982d849130', 'request_id': 'req-bcdb5ce6-c6de-433e-bdbf-51d8ac9e6210'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:33.527803 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bcdb5ce6-c6de-433e-bdbf-51d8ac9e6210 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aeb1da0ee5d47b5a40efd982d849130', 'request_id': 'req-bcdb5ce6-c6de-433e-bdbf-51d8ac9e6210'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:33.540617 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bcdb5ce6-c6de-433e-bdbf-51d8ac9e6210 admin admin] The token cache is being invalidate because project 2aeb1da0ee5d47b5a40efd982d849130 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:35:33.540617 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bcdb5ce6-c6de-433e-bdbf-51d8ac9e6210 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:33.541262 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bcdb5ce6-c6de-433e-bdbf-51d8ac9e6210 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:33.542822 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1073/2156] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:35:33 2022] DELETE /identity/v3/projects/2aeb1da0ee5d47b5a40efd982d849130 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:35:33.548125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e7bc1-0a09-40df-850f-ca9d48c5937b None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:33.548410 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e7bc1-0a09-40df-850f-ca9d48c5937b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:33.548693 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e7bc1-0a09-40df-850f-ca9d48c5937b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:33.572497 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-0d0e7bc1-0a09-40df-850f-ca9d48c5937b None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:33.572497 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-0d0e7bc1-0a09-40df-850f-ca9d48c5937b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:33.633731 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1084/2157] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:35:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:35:33.641688 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466f809-092d-4992-b6ce-ef5a47c562e2 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:33.642194 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466f809-092d-4992-b6ce-ef5a47c562e2 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:33.642194 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466f809-092d-4992-b6ce-ef5a47c562e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:33.661652 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-a466f809-092d-4992-b6ce-ef5a47c562e2 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:33.662875 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-a466f809-092d-4992-b6ce-ef5a47c562e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:33.671427 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1074/2158] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:35:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:35:33.675695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e7bc1-0a09-40df-850f-ca9d48c5937b None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:33.686095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e7bc1-0a09-40df-850f-ca9d48c5937b None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:33.690981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ea4a6-1edb-4027-ae9d-27cd993b9515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:33.691655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ea4a6-1edb-4027-ae9d-27cd993b9515 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:33.691939 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ea4a6-1edb-4027-ae9d-27cd993b9515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:33.692247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ea4a6-1edb-4027-ae9d-27cd993b9515 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:33.693000 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ea4a6-1edb-4027-ae9d-27cd993b9515 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:35:33.693306 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ea4a6-1edb-4027-ae9d-27cd993b9515 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:33.695531 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ea4a6-1edb-4027-ae9d-27cd993b9515 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:33.711256 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1085/2159] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:35:33.715722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a466f809-092d-4992-b6ce-ef5a47c562e2 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:33.725048 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a466f809-092d-4992-b6ce-ef5a47c562e2 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:33.730171 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2851f8-9ebb-49ad-ae36-cdec792bc3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:33.730829 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2851f8-9ebb-49ad-ae36-cdec792bc3a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:33.731112 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2851f8-9ebb-49ad-ae36-cdec792bc3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:33.731388 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2851f8-9ebb-49ad-ae36-cdec792bc3a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:33.732085 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2851f8-9ebb-49ad-ae36-cdec792bc3a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:33.734038 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2851f8-9ebb-49ad-ae36-cdec792bc3a7 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:33.761687 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1075/2160] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:35:33 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:35:33.765906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ea4a6-1edb-4027-ae9d-27cd993b9515 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:33.777005 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ea4a6-1edb-4027-ae9d-27cd993b9515 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:33.854801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1baf3b-2a0c-472d-9b38-88745b8e5320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:33.855488 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1baf3b-2a0c-472d-9b38-88745b8e5320 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:33.855778 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1baf3b-2a0c-472d-9b38-88745b8e5320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:33.856059 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1baf3b-2a0c-472d-9b38-88745b8e5320 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:33.856796 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1baf3b-2a0c-472d-9b38-88745b8e5320 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:33.858766 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1baf3b-2a0c-472d-9b38-88745b8e5320 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:33.875167 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:33.881238 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1086/2161] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:35:33 2022] POST /identity/v3/users => generated 291 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:35:33.887259 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2851f8-9ebb-49ad-ae36-cdec792bc3a7 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:33.895154 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:33.900189 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2851f8-9ebb-49ad-ae36-cdec792bc3a7 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:33.906691 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e083d7-6be5-4348-8207-86eac926ed24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:33.907441 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e083d7-6be5-4348-8207-86eac926ed24 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:33.907742 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e083d7-6be5-4348-8207-86eac926ed24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:33.908223 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e083d7-6be5-4348-8207-86eac926ed24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:33.909023 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e083d7-6be5-4348-8207-86eac926ed24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:33.911043 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e083d7-6be5-4348-8207-86eac926ed24 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:33.923089 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1076/2162] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:35:33.927259 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1baf3b-2a0c-472d-9b38-88745b8e5320 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:33.938062 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1baf3b-2a0c-472d-9b38-88745b8e5320 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:33.944114 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebdf72d-9a8a-4c23-b46b-309365a0ab51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:33.944853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebdf72d-9a8a-4c23-b46b-309365a0ab51 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:33.945144 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebdf72d-9a8a-4c23-b46b-309365a0ab51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:33.945432 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebdf72d-9a8a-4c23-b46b-309365a0ab51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:33.946161 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebdf72d-9a8a-4c23-b46b-309365a0ab51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:33.947765 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebdf72d-9a8a-4c23-b46b-309365a0ab51 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:33.959918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1087/2163] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:35:33.965270 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e083d7-6be5-4348-8207-86eac926ed24 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:33.975800 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e083d7-6be5-4348-8207-86eac926ed24 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:33.981261 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88f569-b8f1-444f-93da-6ab5445e0054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:33.982156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88f569-b8f1-444f-93da-6ab5445e0054 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:33.982438 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88f569-b8f1-444f-93da-6ab5445e0054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:33.982720 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88f569-b8f1-444f-93da-6ab5445e0054 admin admin] PATH_INFO: `/v3/projects/1a4bfb428a4449dbb0d2105a2c1d95f9/users/ce7e308f19c74feab73aabd1d79a072f/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:33.998016 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b88f569-b8f1-444f-93da-6ab5445e0054 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a4bfb428a4449dbb0d2105a2c1d95f9, user_id=ce7e308f19c74feab73aabd1d79a072f, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:34.000736 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b88f569-b8f1-444f-93da-6ab5445e0054 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:34.015900 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1077/2164] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:35:33 2022] PUT /identity/v3/projects/1a4bfb428a4449dbb0d2105a2c1d95f9/users/ce7e308f19c74feab73aabd1d79a072f/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:34.032208 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f0e9c-1b89-4ea7-b9a1-2727761a0aab None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:34.032636 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f0e9c-1b89-4ea7-b9a1-2727761a0aab None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:34.033091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f0e9c-1b89-4ea7-b9a1-2727761a0aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:34.063698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-c91f0e9c-1b89-4ea7-b9a1-2727761a0aab None None] MFA Rules not processed for user `ce7e308f19c74feab73aabd1d79a072f`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:34.064989 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c91f0e9c-1b89-4ea7-b9a1-2727761a0aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:34.134816 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1088/2165] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:35:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:35:34.141229 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88f569-b8f1-444f-93da-6ab5445e0054 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:34.211883 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88f569-b8f1-444f-93da-6ab5445e0054 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:34.216642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a632f-d103-4d2a-8d4c-5a3dace3dfd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:34.217314 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a632f-d103-4d2a-8d4c-5a3dace3dfd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:34.217582 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a632f-d103-4d2a-8d4c-5a3dace3dfd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:34.217857 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a632f-d103-4d2a-8d4c-5a3dace3dfd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:34.218556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287a632f-d103-4d2a-8d4c-5a3dace3dfd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:34.220510 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287a632f-d103-4d2a-8d4c-5a3dace3dfd3 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:34.250029 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1078/2166] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:35:34 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:35:34.251246 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f0e9c-1b89-4ea7-b9a1-2727761a0aab None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:34.262112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f0e9c-1b89-4ea7-b9a1-2727761a0aab None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:34.318888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e4ab5f-bdc1-4441-a5d4-5aef7b9803af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:34.319570 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e4ab5f-bdc1-4441-a5d4-5aef7b9803af admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:34.320404 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e4ab5f-bdc1-4441-a5d4-5aef7b9803af admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:34.320687 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e4ab5f-bdc1-4441-a5d4-5aef7b9803af admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:34.321545 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e4ab5f-bdc1-4441-a5d4-5aef7b9803af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:34.323522 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e4ab5f-bdc1-4441-a5d4-5aef7b9803af admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:34.347948 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1089/2167] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:35:34 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:35:34.349148 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a632f-d103-4d2a-8d4c-5a3dace3dfd3 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:34.359775 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a632f-d103-4d2a-8d4c-5a3dace3dfd3 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:34.364453 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240a5ca-3a1d-4740-a13f-597bb50f69dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:34.365186 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240a5ca-3a1d-4740-a13f-597bb50f69dc admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:34.365468 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240a5ca-3a1d-4740-a13f-597bb50f69dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:34.365747 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240a5ca-3a1d-4740-a13f-597bb50f69dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:34.366466 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240a5ca-3a1d-4740-a13f-597bb50f69dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:34.368072 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240a5ca-3a1d-4740-a13f-597bb50f69dc admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:34.379034 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1079/2168] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:35:34.383852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e4ab5f-bdc1-4441-a5d4-5aef7b9803af admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:34.393943 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e4ab5f-bdc1-4441-a5d4-5aef7b9803af admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:34.399295 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd89cdf-744d-4003-a474-e8418d449410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:34.400256 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd89cdf-744d-4003-a474-e8418d449410 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:34.400495 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd89cdf-744d-4003-a474-e8418d449410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:34.400776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd89cdf-744d-4003-a474-e8418d449410 admin admin] PATH_INFO: `/v3/projects/250b936eb89e43a296ab256cf2a3bf3e/users/54bc1fce625549dab2e18afb6aeb8769/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:34.416485 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd89cdf-744d-4003-a474-e8418d449410 admin admin] RBAC: Authorizing `identity:create_grant(project_id=250b936eb89e43a296ab256cf2a3bf3e, user_id=54bc1fce625549dab2e18afb6aeb8769, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:34.419156 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd89cdf-744d-4003-a474-e8418d449410 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:34.437553 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1090/2169] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:35:34 2022] PUT /identity/v3/projects/250b936eb89e43a296ab256cf2a3bf3e/users/54bc1fce625549dab2e18afb6aeb8769/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:34.441790 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240a5ca-3a1d-4740-a13f-597bb50f69dc admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:34.453689 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240a5ca-3a1d-4740-a13f-597bb50f69dc admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:34.511975 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ba72e-78e8-4bff-a061-65cb42d68b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:34.512889 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ba72e-78e8-4bff-a061-65cb42d68b09 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:34.513202 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ba72e-78e8-4bff-a061-65cb42d68b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:34.513470 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ba72e-78e8-4bff-a061-65cb42d68b09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:34.514210 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0ba72e-78e8-4bff-a061-65cb42d68b09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:34.515822 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0ba72e-78e8-4bff-a061-65cb42d68b09 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:34.528496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1080/2170] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:35:34.531491 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd89cdf-744d-4003-a474-e8418d449410 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:34.543270 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd89cdf-744d-4003-a474-e8418d449410 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:34.548185 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f798f81c-5fcb-4d1d-9247-0a7ed98ec5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:34.548887 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798f81c-5fcb-4d1d-9247-0a7ed98ec5f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:34.549236 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798f81c-5fcb-4d1d-9247-0a7ed98ec5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:34.549580 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f798f81c-5fcb-4d1d-9247-0a7ed98ec5f7 admin admin] PATH_INFO: `/v3/projects/250b936eb89e43a296ab256cf2a3bf3e/users/54bc1fce625549dab2e18afb6aeb8769/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:34.551981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f798f81c-5fcb-4d1d-9247-0a7ed98ec5f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=250b936eb89e43a296ab256cf2a3bf3e, user_id=54bc1fce625549dab2e18afb6aeb8769, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:34.555037 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f798f81c-5fcb-4d1d-9247-0a7ed98ec5f7 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:34.566776 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1091/2171] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:35:34 2022] PUT /identity/v3/projects/250b936eb89e43a296ab256cf2a3bf3e/users/54bc1fce625549dab2e18afb6aeb8769/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:34.577704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85988c1a-dd86-4901-b866-b085927d646e None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:34.577997 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85988c1a-dd86-4901-b866-b085927d646e None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:34.578310 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85988c1a-dd86-4901-b866-b085927d646e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:34.608947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-85988c1a-dd86-4901-b866-b085927d646e None None] MFA Rules not processed for user `54bc1fce625549dab2e18afb6aeb8769`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:34.611723 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-85988c1a-dd86-4901-b866-b085927d646e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:34.647026 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:34.689678 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1081/2172] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:35:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:35:34.702846 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f798f81c-5fcb-4d1d-9247-0a7ed98ec5f7 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:34.704615 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-f798f81c-5fcb-4d1d-9247-0a7ed98ec5f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:34.740573 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:34.767383 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f798f81c-5fcb-4d1d-9247-0a7ed98ec5f7 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:34.771075 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4893f-becb-49f9-9779-8ca96eff61a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:34.771733 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4893f-becb-49f9-9779-8ca96eff61a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:34.772023 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4893f-becb-49f9-9779-8ca96eff61a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:34.772347 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4893f-becb-49f9-9779-8ca96eff61a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:34.782079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f4893f-becb-49f9-9779-8ca96eff61a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:34.783559 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f4893f-becb-49f9-9779-8ca96eff61a6 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:34.793664 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1092/2173] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:35:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:35:35.141282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85988c1a-dd86-4901-b866-b085927d646e None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:35.152114 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85988c1a-dd86-4901-b866-b085927d646e None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:35.157613 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4ec9f-e89c-4886-b6c3-d39f3699f764 tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bc1fce625549dab2e18afb6aeb8769', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '250b936eb89e43a296ab256cf2a3bf3e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:35.158319 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4ec9f-e89c-4886-b6c3-d39f3699f764 tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:35.158655 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4ec9f-e89c-4886-b6c3-d39f3699f764 tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:35.159231 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4ec9f-e89c-4886-b6c3-d39f3699f764 tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:35.160150 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a4ec9f-e89c-4886-b6c3-d39f3699f764 tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:35:35.160736 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a4ec9f-e89c-4886-b6c3-d39f3699f764 tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:35.163739 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a4ec9f-e89c-4886-b6c3-d39f3699f764 tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:35.176954 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1082/2174] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:35:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:35:35.180624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4893f-becb-49f9-9779-8ca96eff61a6 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:35.190323 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4893f-becb-49f9-9779-8ca96eff61a6 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:35.196697 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b476fc-dfe2-4110-a7d4-ff2862142f9b tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bc1fce625549dab2e18afb6aeb8769', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '250b936eb89e43a296ab256cf2a3bf3e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:35.197373 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b476fc-dfe2-4110-a7d4-ff2862142f9b tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:35.197662 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b476fc-dfe2-4110-a7d4-ff2862142f9b tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:35.198003 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b476fc-dfe2-4110-a7d4-ff2862142f9b tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:35.198652 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b476fc-dfe2-4110-a7d4-ff2862142f9b tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:35.200591 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b476fc-dfe2-4110-a7d4-ff2862142f9b tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:35.248485 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1093/2175] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:35:35 2022] POST /identity/v3/projects => generated 353 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:35:35.252716 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4ec9f-e89c-4886-b6c3-d39f3699f764 tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:35.263837 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4ec9f-e89c-4886-b6c3-d39f3699f764 tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:35.324791 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2613c-603a-4070-bef6-5f766b961c6a tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bc1fce625549dab2e18afb6aeb8769', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '250b936eb89e43a296ab256cf2a3bf3e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:35.325470 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2613c-603a-4070-bef6-5f766b961c6a tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:35.325765 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2613c-603a-4070-bef6-5f766b961c6a tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:35.326335 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2613c-603a-4070-bef6-5f766b961c6a tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:35.327076 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2613c-603a-4070-bef6-5f766b961c6a tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:35:35.327425 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2613c-603a-4070-bef6-5f766b961c6a tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:35.329084 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f2613c-603a-4070-bef6-5f766b961c6a tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:35.361288 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1083/2176] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:35:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:35:35.404586 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b476fc-dfe2-4110-a7d4-ff2862142f9b tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:35.422959 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b476fc-dfe2-4110-a7d4-ff2862142f9b tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:35.431625 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82dda2-322b-405b-94f5-4e2b10ac3ff5 tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54bc1fce625549dab2e18afb6aeb8769', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '250b936eb89e43a296ab256cf2a3bf3e', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:35.434172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82dda2-322b-405b-94f5-4e2b10ac3ff5 tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:35.434689 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82dda2-322b-405b-94f5-4e2b10ac3ff5 tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:35.435201 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82dda2-322b-405b-94f5-4e2b10ac3ff5 tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] PATH_INFO: `/v3/projects/57d37def253f4f3c86a53f601cc5c4e0` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:35.438604 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab82dda2-322b-405b-94f5-4e2b10ac3ff5 tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] RBAC: Authorizing `identity:delete_project(project_id=57d37def253f4f3c86a53f601cc5c4e0)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:35.442481 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab82dda2-322b-405b-94f5-4e2b10ac3ff5 tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:35.514528 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ab82dda2-322b-405b-94f5-4e2b10ac3ff5 tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57d37def253f4f3c86a53f601cc5c4e0', 'request_id': 'req-ab82dda2-322b-405b-94f5-4e2b10ac3ff5'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:35.520449 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ab82dda2-322b-405b-94f5-4e2b10ac3ff5 tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57d37def253f4f3c86a53f601cc5c4e0', 'request_id': 'req-ab82dda2-322b-405b-94f5-4e2b10ac3ff5'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:35.527064 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ab82dda2-322b-405b-94f5-4e2b10ac3ff5 tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57d37def253f4f3c86a53f601cc5c4e0', 'request_id': 'req-ab82dda2-322b-405b-94f5-4e2b10ac3ff5'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:35.541190 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ab82dda2-322b-405b-94f5-4e2b10ac3ff5 tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] The token cache is being invalidate because project 57d37def253f4f3c86a53f601cc5c4e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:35:35.541392 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ab82dda2-322b-405b-94f5-4e2b10ac3ff5 tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:35.542093 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ab82dda2-322b-405b-94f5-4e2b10ac3ff5 tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:35.543867 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1094/2177] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:35:35 2022] DELETE /identity/v3/projects/57d37def253f4f3c86a53f601cc5c4e0 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:35.765562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2613c-603a-4070-bef6-5f766b961c6a tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:35.767205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-53f2613c-603a-4070-bef6-5f766b961c6a tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:35.832135 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2613c-603a-4070-bef6-5f766b961c6a tempest-VolumeQuotaClassesTest-1737729735 tempest-VolumeQuotaClassesTest-1737729735-project-admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:35.883093 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:35.891777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0a3c6-d68e-42b7-b537-ca380af18883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:35.892708 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0a3c6-d68e-42b7-b537-ca380af18883 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:35.892993 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0a3c6-d68e-42b7-b537-ca380af18883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:35.893270 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0a3c6-d68e-42b7-b537-ca380af18883 admin admin] PATH_INFO: `/v3/users/ce7e308f19c74feab73aabd1d79a072f` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:35.894603 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f0a3c6-d68e-42b7-b537-ca380af18883 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce7e308f19c74feab73aabd1d79a072f)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:35.896442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f0a3c6-d68e-42b7-b537-ca380af18883 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:35.948221 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-37f0a3c6-d68e-42b7-b537-ca380af18883 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce7e308f19c74feab73aabd1d79a072f'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:35.948889 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-37f0a3c6-d68e-42b7-b537-ca380af18883 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce7e308f19c74feab73aabd1d79a072f'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:35.957788 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-37f0a3c6-d68e-42b7-b537-ca380af18883 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce7e308f19c74feab73aabd1d79a072f'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:35.973169 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-37f0a3c6-d68e-42b7-b537-ca380af18883 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce7e308f19c74feab73aabd1d79a072f'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:35.983392 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-37f0a3c6-d68e-42b7-b537-ca380af18883 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce7e308f19c74feab73aabd1d79a072f'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:35.990802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1084/2178] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:35:35 2022] DELETE /identity/v3/users/ce7e308f19c74feab73aabd1d79a072f => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:35.996539 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf09266-6d6c-49e3-8bbe-fd75e1e388d7 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:35.996954 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf09266-6d6c-49e3-8bbe-fd75e1e388d7 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:35.997339 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf09266-6d6c-49e3-8bbe-fd75e1e388d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:36.015224 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-dcf09266-6d6c-49e3-8bbe-fd75e1e388d7 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:36.016681 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-dcf09266-6d6c-49e3-8bbe-fd75e1e388d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:36.081070 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1095/2179] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:35:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:35:36.090443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0a3c6-d68e-42b7-b537-ca380af18883 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:36.091810 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-37f0a3c6-d68e-42b7-b537-ca380af18883 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:36.148124 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0a3c6-d68e-42b7-b537-ca380af18883 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:36.151791 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944dc55e-d0a6-4ac2-a249-612fe0194e1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:36.152467 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944dc55e-d0a6-4ac2-a249-612fe0194e1c service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:36.152754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944dc55e-d0a6-4ac2-a249-612fe0194e1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:36.153035 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944dc55e-d0a6-4ac2-a249-612fe0194e1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:36.161930 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944dc55e-d0a6-4ac2-a249-612fe0194e1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:36.163360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944dc55e-d0a6-4ac2-a249-612fe0194e1c service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:36.173406 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1085/2180] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:35:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:35:36.468758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf09266-6d6c-49e3-8bbe-fd75e1e388d7 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:36.470330 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-dcf09266-6d6c-49e3-8bbe-fd75e1e388d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:36.483379 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf09266-6d6c-49e3-8bbe-fd75e1e388d7 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:36.488083 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3de49-5de8-4a27-a20a-81eace12982b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:36.489019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3de49-5de8-4a27-a20a-81eace12982b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:36.489293 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3de49-5de8-4a27-a20a-81eace12982b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:36.489569 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3de49-5de8-4a27-a20a-81eace12982b admin admin] PATH_INFO: `/v3/projects/1a4bfb428a4449dbb0d2105a2c1d95f9` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:36.490777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a3de49-5de8-4a27-a20a-81eace12982b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a4bfb428a4449dbb0d2105a2c1d95f9)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:36.492792 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a3de49-5de8-4a27-a20a-81eace12982b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:36.527896 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:36.542214 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-03a3de49-5de8-4a27-a20a-81eace12982b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a4bfb428a4449dbb0d2105a2c1d95f9', 'request_id': 'req-03a3de49-5de8-4a27-a20a-81eace12982b'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:36.547684 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-03a3de49-5de8-4a27-a20a-81eace12982b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a4bfb428a4449dbb0d2105a2c1d95f9', 'request_id': 'req-03a3de49-5de8-4a27-a20a-81eace12982b'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:36.553747 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-03a3de49-5de8-4a27-a20a-81eace12982b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a4bfb428a4449dbb0d2105a2c1d95f9', 'request_id': 'req-03a3de49-5de8-4a27-a20a-81eace12982b'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:36.565562 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-03a3de49-5de8-4a27-a20a-81eace12982b admin admin] The token cache is being invalidate because project 1a4bfb428a4449dbb0d2105a2c1d95f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:35:36.565848 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-03a3de49-5de8-4a27-a20a-81eace12982b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:36.566460 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-03a3de49-5de8-4a27-a20a-81eace12982b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:36.568005 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1096/2181] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:35:36 2022] DELETE /identity/v3/projects/1a4bfb428a4449dbb0d2105a2c1d95f9 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:35:36.573076 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944dc55e-d0a6-4ac2-a249-612fe0194e1c service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:36.582811 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944dc55e-d0a6-4ac2-a249-612fe0194e1c service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:36.638624 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f442b-6eed-4a24-a528-2182cc83a232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:36.639491 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f442b-6eed-4a24-a528-2182cc83a232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:36.639778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f442b-6eed-4a24-a528-2182cc83a232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:36.640060 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f442b-6eed-4a24-a528-2182cc83a232 admin admin] PATH_INFO: `/v3/users/54bc1fce625549dab2e18afb6aeb8769` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:36.641286 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f442b-6eed-4a24-a528-2182cc83a232 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54bc1fce625549dab2e18afb6aeb8769)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:36.643209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f442b-6eed-4a24-a528-2182cc83a232 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:36.660005 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:36.697057 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-663f442b-6eed-4a24-a528-2182cc83a232 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54bc1fce625549dab2e18afb6aeb8769'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:36.697797 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-663f442b-6eed-4a24-a528-2182cc83a232 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54bc1fce625549dab2e18afb6aeb8769'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:36.706896 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-663f442b-6eed-4a24-a528-2182cc83a232 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54bc1fce625549dab2e18afb6aeb8769'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:36.721615 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-663f442b-6eed-4a24-a528-2182cc83a232 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54bc1fce625549dab2e18afb6aeb8769'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:36.729532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-663f442b-6eed-4a24-a528-2182cc83a232 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54bc1fce625549dab2e18afb6aeb8769'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:36.736209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1086/2182] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:35:36 2022] DELETE /identity/v3/users/54bc1fce625549dab2e18afb6aeb8769 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:35:37.135617 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3de49-5de8-4a27-a20a-81eace12982b admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:37.138821 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-03a3de49-5de8-4a27-a20a-81eace12982b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:37.171150 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3de49-5de8-4a27-a20a-81eace12982b admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:37.249965 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41217d4a-9a81-47e2-9262-e3f3c07e9733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:37.250896 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41217d4a-9a81-47e2-9262-e3f3c07e9733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:37.251181 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41217d4a-9a81-47e2-9262-e3f3c07e9733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:37.251462 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41217d4a-9a81-47e2-9262-e3f3c07e9733 admin admin] PATH_INFO: `/v3/projects/250b936eb89e43a296ab256cf2a3bf3e` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:37.252742 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41217d4a-9a81-47e2-9262-e3f3c07e9733 admin admin] RBAC: Authorizing `identity:delete_project(project_id=250b936eb89e43a296ab256cf2a3bf3e)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:37.254784 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41217d4a-9a81-47e2-9262-e3f3c07e9733 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:37.312825 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-41217d4a-9a81-47e2-9262-e3f3c07e9733 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '250b936eb89e43a296ab256cf2a3bf3e', 'request_id': 'req-41217d4a-9a81-47e2-9262-e3f3c07e9733'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:37.318876 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-41217d4a-9a81-47e2-9262-e3f3c07e9733 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '250b936eb89e43a296ab256cf2a3bf3e', 'request_id': 'req-41217d4a-9a81-47e2-9262-e3f3c07e9733'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:37.325756 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-41217d4a-9a81-47e2-9262-e3f3c07e9733 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '250b936eb89e43a296ab256cf2a3bf3e', 'request_id': 'req-41217d4a-9a81-47e2-9262-e3f3c07e9733'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:37.342485 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-41217d4a-9a81-47e2-9262-e3f3c07e9733 admin admin] The token cache is being invalidate because project 250b936eb89e43a296ab256cf2a3bf3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:35:37.342782 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-41217d4a-9a81-47e2-9262-e3f3c07e9733 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:37.343440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-41217d4a-9a81-47e2-9262-e3f3c07e9733 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:37.345067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1097/2183] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:35:37 2022] DELETE /identity/v3/projects/250b936eb89e43a296ab256cf2a3bf3e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:37.352578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231f3e3-38b0-4a7a-afb2-37f487cf30e4 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:37.353062 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231f3e3-38b0-4a7a-afb2-37f487cf30e4 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:37.353177 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231f3e3-38b0-4a7a-afb2-37f487cf30e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:37.373615 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-a231f3e3-38b0-4a7a-afb2-37f487cf30e4 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:37.374826 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-a231f3e3-38b0-4a7a-afb2-37f487cf30e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:37.437627 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1087/2184] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:35:37.447089 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98553449-dbab-4f17-a318-3ddfd9f89d10 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:37.447384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98553449-dbab-4f17-a318-3ddfd9f89d10 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:37.447670 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98553449-dbab-4f17-a318-3ddfd9f89d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:37.468575 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-98553449-dbab-4f17-a318-3ddfd9f89d10 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:37.469801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-98553449-dbab-4f17-a318-3ddfd9f89d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:37.478210 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1098/2185] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:35:37.482329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a231f3e3-38b0-4a7a-afb2-37f487cf30e4 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:37.491874 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a231f3e3-38b0-4a7a-afb2-37f487cf30e4 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:37.496516 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4307f-bd27-4a3d-a931-c964127b9ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:37.497179 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4307f-bd27-4a3d-a931-c964127b9ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:37.497601 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4307f-bd27-4a3d-a931-c964127b9ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:37.497750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4307f-bd27-4a3d-a931-c964127b9ea7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:37.498451 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4307f-bd27-4a3d-a931-c964127b9ea7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:35:37.498762 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4307f-bd27-4a3d-a931-c964127b9ea7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:37.500392 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4307f-bd27-4a3d-a931-c964127b9ea7 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:37.514128 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1088/2186] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:35:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:35:37.521345 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98553449-dbab-4f17-a318-3ddfd9f89d10 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:37.529538 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98553449-dbab-4f17-a318-3ddfd9f89d10 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:37.534679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9074a28-b111-4e10-b362-0864b45c38d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:37.535478 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9074a28-b111-4e10-b362-0864b45c38d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:37.535857 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9074a28-b111-4e10-b362-0864b45c38d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:37.536244 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9074a28-b111-4e10-b362-0864b45c38d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:37.537283 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9074a28-b111-4e10-b362-0864b45c38d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:37.540444 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9074a28-b111-4e10-b362-0864b45c38d0 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:37.573720 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1099/2187] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:35:37 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:35:37.578945 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4307f-bd27-4a3d-a931-c964127b9ea7 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:37.595611 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4307f-bd27-4a3d-a931-c964127b9ea7 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:37.657180 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42d3df-cf07-48b1-b3c7-281314b6f5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:37.657906 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42d3df-cf07-48b1-b3c7-281314b6f5d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:37.658227 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42d3df-cf07-48b1-b3c7-281314b6f5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:37.658578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42d3df-cf07-48b1-b3c7-281314b6f5d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:37.659312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f42d3df-cf07-48b1-b3c7-281314b6f5d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:37.661300 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f42d3df-cf07-48b1-b3c7-281314b6f5d0 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:37.685605 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1089/2188] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:35:37 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:35:37.686860 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9074a28-b111-4e10-b362-0864b45c38d0 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:37.698015 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9074a28-b111-4e10-b362-0864b45c38d0 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:37.703191 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fafab5-7d8b-4f69-8384-869441f5f9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:37.703864 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fafab5-7d8b-4f69-8384-869441f5f9ce admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:37.704146 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fafab5-7d8b-4f69-8384-869441f5f9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:37.704461 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fafab5-7d8b-4f69-8384-869441f5f9ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:37.705187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fafab5-7d8b-4f69-8384-869441f5f9ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:37.706807 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fafab5-7d8b-4f69-8384-869441f5f9ce admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:37.724955 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1100/2189] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:35:37.725732 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42d3df-cf07-48b1-b3c7-281314b6f5d0 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:37.737712 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42d3df-cf07-48b1-b3c7-281314b6f5d0 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:37.745543 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19549fb6-25b3-40b3-bd71-5d7893c4408e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:37.746807 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19549fb6-25b3-40b3-bd71-5d7893c4408e admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:37.747272 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19549fb6-25b3-40b3-bd71-5d7893c4408e admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:37.747810 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19549fb6-25b3-40b3-bd71-5d7893c4408e admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:37.749081 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19549fb6-25b3-40b3-bd71-5d7893c4408e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:37.751647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19549fb6-25b3-40b3-bd71-5d7893c4408e admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:37.758914 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:37.765844 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1090/2190] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:35:37.770029 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fafab5-7d8b-4f69-8384-869441f5f9ce admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:37.780397 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fafab5-7d8b-4f69-8384-869441f5f9ce admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:37.785648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c4e16-b034-4d55-9f6e-520b2839e3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:37.786540 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c4e16-b034-4d55-9f6e-520b2839e3ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:37.786821 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c4e16-b034-4d55-9f6e-520b2839e3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:37.787099 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c4e16-b034-4d55-9f6e-520b2839e3ef admin admin] PATH_INFO: `/v3/projects/992fa1b6160f48f59ae18307256b6f72/users/cb64ee7ac8c54403926403b1e6dfbcad/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:37.791421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:37.804019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c4e16-b034-4d55-9f6e-520b2839e3ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=992fa1b6160f48f59ae18307256b6f72, user_id=cb64ee7ac8c54403926403b1e6dfbcad, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:37.807177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c4e16-b034-4d55-9f6e-520b2839e3ef admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:37.821197 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1101/2191] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:35:37 2022] PUT /identity/v3/projects/992fa1b6160f48f59ae18307256b6f72/users/cb64ee7ac8c54403926403b1e6dfbcad/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:37.832058 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12894af-4e94-455f-b74a-3ce4fe22dac1 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:37.832369 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12894af-4e94-455f-b74a-3ce4fe22dac1 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:37.832655 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12894af-4e94-455f-b74a-3ce4fe22dac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:37.864554 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-e12894af-4e94-455f-b74a-3ce4fe22dac1 None None] MFA Rules not processed for user `cb64ee7ac8c54403926403b1e6dfbcad`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:37.865786 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-e12894af-4e94-455f-b74a-3ce4fe22dac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:37.939845 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1091/2192] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:35:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:35:37.945782 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c4e16-b034-4d55-9f6e-520b2839e3ef admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:38.010456 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c4e16-b034-4d55-9f6e-520b2839e3ef admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:38.015649 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0b6bc-12a2-4c8e-97c3-9934f6c64871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:38.016022 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0b6bc-12a2-4c8e-97c3-9934f6c64871 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:38.016343 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0b6bc-12a2-4c8e-97c3-9934f6c64871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:38.016628 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0b6bc-12a2-4c8e-97c3-9934f6c64871 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:38.017380 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc0b6bc-12a2-4c8e-97c3-9934f6c64871 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:38.023909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc0b6bc-12a2-4c8e-97c3-9934f6c64871 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:38.052831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1102/2193] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:35:37 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:35:38.054204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12894af-4e94-455f-b74a-3ce4fe22dac1 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:38.066336 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12894af-4e94-455f-b74a-3ce4fe22dac1 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:38.128733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c9d6a-1d2e-4898-9cb8-51d1f7a0e61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:38.129979 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c9d6a-1d2e-4898-9cb8-51d1f7a0e61e admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:38.129979 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c9d6a-1d2e-4898-9cb8-51d1f7a0e61e admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:38.129979 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c9d6a-1d2e-4898-9cb8-51d1f7a0e61e admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:38.130629 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58c9d6a-1d2e-4898-9cb8-51d1f7a0e61e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:38.132658 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58c9d6a-1d2e-4898-9cb8-51d1f7a0e61e admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:38.164770 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1092/2194] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:35:38 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:35:38.166178 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0b6bc-12a2-4c8e-97c3-9934f6c64871 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:38.176512 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0b6bc-12a2-4c8e-97c3-9934f6c64871 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:38.181964 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10cee67-9aa8-4ee0-a148-7e81391998f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:38.182636 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10cee67-9aa8-4ee0-a148-7e81391998f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:38.182923 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10cee67-9aa8-4ee0-a148-7e81391998f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:38.183198 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10cee67-9aa8-4ee0-a148-7e81391998f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:38.183921 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10cee67-9aa8-4ee0-a148-7e81391998f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:38.185595 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10cee67-9aa8-4ee0-a148-7e81391998f6 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:38.196967 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1103/2195] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:35:38.200975 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c9d6a-1d2e-4898-9cb8-51d1f7a0e61e admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:38.211520 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c9d6a-1d2e-4898-9cb8-51d1f7a0e61e admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:38.216247 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a92463-ff9a-4ef9-8051-f6306e8a606a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:38.216919 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a92463-ff9a-4ef9-8051-f6306e8a606a admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:38.217208 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a92463-ff9a-4ef9-8051-f6306e8a606a admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:38.217488 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a92463-ff9a-4ef9-8051-f6306e8a606a admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:38.218226 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a92463-ff9a-4ef9-8051-f6306e8a606a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:38.219840 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a92463-ff9a-4ef9-8051-f6306e8a606a admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:38.232045 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1093/2196] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:35:38.236330 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10cee67-9aa8-4ee0-a148-7e81391998f6 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:38.247524 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10cee67-9aa8-4ee0-a148-7e81391998f6 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:38.252594 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2fc2c1-4eca-4bd5-bd6a-9212b79558a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:38.253686 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2fc2c1-4eca-4bd5-bd6a-9212b79558a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:38.253801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2fc2c1-4eca-4bd5-bd6a-9212b79558a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:38.254091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2fc2c1-4eca-4bd5-bd6a-9212b79558a7 admin admin] PATH_INFO: `/v3/projects/d0be3eb87853483a9e214f43cc520b7b/users/582ef8464e914e379d913747ae2d1e17/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:38.270312 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2fc2c1-4eca-4bd5-bd6a-9212b79558a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0be3eb87853483a9e214f43cc520b7b, user_id=582ef8464e914e379d913747ae2d1e17, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:38.273080 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2fc2c1-4eca-4bd5-bd6a-9212b79558a7 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:38.297532 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1104/2197] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:35:38 2022] PUT /identity/v3/projects/d0be3eb87853483a9e214f43cc520b7b/users/582ef8464e914e379d913747ae2d1e17/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:38.310750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbd25d-f759-49cf-8f3f-0bd4468d4931 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:38.311094 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbd25d-f759-49cf-8f3f-0bd4468d4931 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:38.311502 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbd25d-f759-49cf-8f3f-0bd4468d4931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:38.358971 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-2ffbd25d-f759-49cf-8f3f-0bd4468d4931 None None] MFA Rules not processed for user `582ef8464e914e379d913747ae2d1e17`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:38.361688 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-2ffbd25d-f759-49cf-8f3f-0bd4468d4931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:38.436509 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:38.444217 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1094/2198] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:35:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:35:38.447815 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2fc2c1-4eca-4bd5-bd6a-9212b79558a7 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:38.516818 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2fc2c1-4eca-4bd5-bd6a-9212b79558a7 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:38.522083 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513fc389-3ef9-47aa-a83f-cb2bdd7c7fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:38.522780 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513fc389-3ef9-47aa-a83f-cb2bdd7c7fb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:38.523586 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513fc389-3ef9-47aa-a83f-cb2bdd7c7fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:38.523874 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513fc389-3ef9-47aa-a83f-cb2bdd7c7fb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:38.524722 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513fc389-3ef9-47aa-a83f-cb2bdd7c7fb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:38.526688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513fc389-3ef9-47aa-a83f-cb2bdd7c7fb7 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:38.566238 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1105/2199] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:35:38 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:35:38.567731 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffbd25d-f759-49cf-8f3f-0bd4468d4931 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:38.579399 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffbd25d-f759-49cf-8f3f-0bd4468d4931 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:38.665165 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423e5beb-a275-4ce9-8ff5-8af36835c514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:38.665856 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e5beb-a275-4ce9-8ff5-8af36835c514 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:38.666148 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e5beb-a275-4ce9-8ff5-8af36835c514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:38.666439 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e5beb-a275-4ce9-8ff5-8af36835c514 admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:38.667195 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423e5beb-a275-4ce9-8ff5-8af36835c514 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:38.669276 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423e5beb-a275-4ce9-8ff5-8af36835c514 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:38.692204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1095/2200] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:35:38 2022] POST /identity/v3/users => generated 299 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:35:38.698541 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513fc389-3ef9-47aa-a83f-cb2bdd7c7fb7 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:38.712123 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513fc389-3ef9-47aa-a83f-cb2bdd7c7fb7 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:38.718890 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f468af4-b063-45d2-8f9e-8246d6f262ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:38.719524 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f468af4-b063-45d2-8f9e-8246d6f262ec admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:38.719857 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f468af4-b063-45d2-8f9e-8246d6f262ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:38.720471 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f468af4-b063-45d2-8f9e-8246d6f262ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:38.722067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f468af4-b063-45d2-8f9e-8246d6f262ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:38.723787 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f468af4-b063-45d2-8f9e-8246d6f262ec admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:38.738584 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1106/2201] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:35:38.743534 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423e5beb-a275-4ce9-8ff5-8af36835c514 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:38.758143 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423e5beb-a275-4ce9-8ff5-8af36835c514 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:38.764736 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfa1e61-65d6-4a6d-bc11-2cad424d9127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:38.766159 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa1e61-65d6-4a6d-bc11-2cad424d9127 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:38.766449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa1e61-65d6-4a6d-bc11-2cad424d9127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:38.766734 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa1e61-65d6-4a6d-bc11-2cad424d9127 admin admin] PATH_INFO: `/v3/projects/f8b67aa9f1934f4a9612bd3c419fab57/users/2730479ac8374af7ac435c6ef5a90dd6/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:38.790414 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfa1e61-65d6-4a6d-bc11-2cad424d9127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8b67aa9f1934f4a9612bd3c419fab57, user_id=2730479ac8374af7ac435c6ef5a90dd6, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:38.793720 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfa1e61-65d6-4a6d-bc11-2cad424d9127 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:38.806804 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1096/2202] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:35:38 2022] PUT /identity/v3/projects/f8b67aa9f1934f4a9612bd3c419fab57/users/2730479ac8374af7ac435c6ef5a90dd6/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:38.811150 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f468af4-b063-45d2-8f9e-8246d6f262ec admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:38.821997 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f468af4-b063-45d2-8f9e-8246d6f262ec admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:38.826907 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:38.886320 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44482e89-648f-43cd-a359-9095e3207d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:38.887000 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44482e89-648f-43cd-a359-9095e3207d01 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:38.887285 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44482e89-648f-43cd-a359-9095e3207d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:38.887566 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44482e89-648f-43cd-a359-9095e3207d01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:38.888330 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44482e89-648f-43cd-a359-9095e3207d01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:38.889928 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44482e89-648f-43cd-a359-9095e3207d01 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:38.901025 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1107/2203] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:35:38.909424 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfa1e61-65d6-4a6d-bc11-2cad424d9127 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:38.920568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfa1e61-65d6-4a6d-bc11-2cad424d9127 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:38.925351 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fbc6ce-147b-4200-9d6e-0f17a17df098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:38.926397 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fbc6ce-147b-4200-9d6e-0f17a17df098 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:38.926714 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fbc6ce-147b-4200-9d6e-0f17a17df098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:38.927001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fbc6ce-147b-4200-9d6e-0f17a17df098 admin admin] PATH_INFO: `/v3/projects/f8b67aa9f1934f4a9612bd3c419fab57/users/2730479ac8374af7ac435c6ef5a90dd6/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:38.929120 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fbc6ce-147b-4200-9d6e-0f17a17df098 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8b67aa9f1934f4a9612bd3c419fab57, user_id=2730479ac8374af7ac435c6ef5a90dd6, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:38.931870 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fbc6ce-147b-4200-9d6e-0f17a17df098 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:38.942832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1097/2204] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:35:38 2022] PUT /identity/v3/projects/f8b67aa9f1934f4a9612bd3c419fab57/users/2730479ac8374af7ac435c6ef5a90dd6/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:35:38.954764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f07cb-d7e8-41b9-b912-628109e71d44 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:38.955052 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f07cb-d7e8-41b9-b912-628109e71d44 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:38.955336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21f07cb-d7e8-41b9-b912-628109e71d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:38.984434 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-d21f07cb-d7e8-41b9-b912-628109e71d44 None None] MFA Rules not processed for user `2730479ac8374af7ac435c6ef5a90dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:38.985651 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d21f07cb-d7e8-41b9-b912-628109e71d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:39.056577 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1108/2205] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:35:38 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:35:39.067345 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fbc6ce-147b-4200-9d6e-0f17a17df098 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:39.071364 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-a6fbc6ce-147b-4200-9d6e-0f17a17df098 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:39.127345 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fbc6ce-147b-4200-9d6e-0f17a17df098 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:39.135144 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad33ef03-e0c1-4bc8-b32a-5b822d9bd5d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:39.135808 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad33ef03-e0c1-4bc8-b32a-5b822d9bd5d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:39.136099 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad33ef03-e0c1-4bc8-b32a-5b822d9bd5d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:39.136419 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad33ef03-e0c1-4bc8-b32a-5b822d9bd5d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:39.146228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad33ef03-e0c1-4bc8-b32a-5b822d9bd5d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:39.147682 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad33ef03-e0c1-4bc8-b32a-5b822d9bd5d4 service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:39.158740 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1098/2206] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:35:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:35:39.716018 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21f07cb-d7e8-41b9-b912-628109e71d44 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:39.717489 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d21f07cb-d7e8-41b9-b912-628109e71d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:39.730834 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21f07cb-d7e8-41b9-b912-628109e71d44 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:39.734471 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e31cc-b8be-40b5-a93a-cd5bf1637b91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:39.735119 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e31cc-b8be-40b5-a93a-cd5bf1637b91 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:39.735406 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e31cc-b8be-40b5-a93a-cd5bf1637b91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:39.735686 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e31cc-b8be-40b5-a93a-cd5bf1637b91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:39.796270 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5e31cc-b8be-40b5-a93a-cd5bf1637b91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:39.799102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5e31cc-b8be-40b5-a93a-cd5bf1637b91 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:39.811304 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1109/2207] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:35:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:35:40.961676 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad33ef03-e0c1-4bc8-b32a-5b822d9bd5d4 service cinder] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:40.963129 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-ad33ef03-e0c1-4bc8-b32a-5b822d9bd5d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:41.018937 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:41.040902 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad33ef03-e0c1-4bc8-b32a-5b822d9bd5d4 service cinder] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:41.047698 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4de1d-a77d-41a5-82ed-13a269dab4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:41.048618 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4de1d-a77d-41a5-82ed-13a269dab4c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:41.048907 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4de1d-a77d-41a5-82ed-13a269dab4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:41.049189 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4de1d-a77d-41a5-82ed-13a269dab4c4 admin admin] PATH_INFO: `/v3/users/10a62759fc814b1cb53654ccf99f4749` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:41.050402 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe4de1d-a77d-41a5-82ed-13a269dab4c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10a62759fc814b1cb53654ccf99f4749)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:41.052396 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe4de1d-a77d-41a5-82ed-13a269dab4c4 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:41.109523 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2fe4de1d-a77d-41a5-82ed-13a269dab4c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10a62759fc814b1cb53654ccf99f4749'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:41.110195 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2fe4de1d-a77d-41a5-82ed-13a269dab4c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10a62759fc814b1cb53654ccf99f4749'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:41.117334 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2fe4de1d-a77d-41a5-82ed-13a269dab4c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10a62759fc814b1cb53654ccf99f4749'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:41.129782 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2fe4de1d-a77d-41a5-82ed-13a269dab4c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10a62759fc814b1cb53654ccf99f4749'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:41.141772 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2fe4de1d-a77d-41a5-82ed-13a269dab4c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10a62759fc814b1cb53654ccf99f4749'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:41.151702 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1099/2208] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:35:40 2022] DELETE /identity/v3/users/10a62759fc814b1cb53654ccf99f4749 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:41.153388 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09936933-f18c-4b14-88ae-570654717942 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:41.153773 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09936933-f18c-4b14-88ae-570654717942 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:41.153944 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09936933-f18c-4b14-88ae-570654717942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:41.180573 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-09936933-f18c-4b14-88ae-570654717942 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:41.181790 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-09936933-f18c-4b14-88ae-570654717942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:41.208845 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:41.251216 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1110/2209] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:35:41.261441 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4de1d-a77d-41a5-82ed-13a269dab4c4 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:41.262883 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-2fe4de1d-a77d-41a5-82ed-13a269dab4c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:41.323272 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4de1d-a77d-41a5-82ed-13a269dab4c4 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:41.327663 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0285d8c2-7cb9-4a71-8539-47be376626c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:41.328461 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285d8c2-7cb9-4a71-8539-47be376626c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:41.328734 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285d8c2-7cb9-4a71-8539-47be376626c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:41.329031 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285d8c2-7cb9-4a71-8539-47be376626c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:41.339253 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0285d8c2-7cb9-4a71-8539-47be376626c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:41.340858 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0285d8c2-7cb9-4a71-8539-47be376626c7 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:41.352969 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1100/2210] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:35:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:35:41.636384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09936933-f18c-4b14-88ae-570654717942 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:41.638023 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-09936933-f18c-4b14-88ae-570654717942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:41.663215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09936933-f18c-4b14-88ae-570654717942 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:41.670810 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f5ada-44a2-4cd7-8993-f42907048ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:41.671898 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f5ada-44a2-4cd7-8993-f42907048ae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:41.672286 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f5ada-44a2-4cd7-8993-f42907048ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:41.672627 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f5ada-44a2-4cd7-8993-f42907048ae0 admin admin] PATH_INFO: `/v3/projects/fb08080f82ff4d73a15c8abd7cb8495f` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:41.673883 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f5ada-44a2-4cd7-8993-f42907048ae0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb08080f82ff4d73a15c8abd7cb8495f)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:41.675855 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f5ada-44a2-4cd7-8993-f42907048ae0 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:41.733162 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cc4f5ada-44a2-4cd7-8993-f42907048ae0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb08080f82ff4d73a15c8abd7cb8495f', 'request_id': 'req-cc4f5ada-44a2-4cd7-8993-f42907048ae0'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:41.739704 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cc4f5ada-44a2-4cd7-8993-f42907048ae0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb08080f82ff4d73a15c8abd7cb8495f', 'request_id': 'req-cc4f5ada-44a2-4cd7-8993-f42907048ae0'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:41.748264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cc4f5ada-44a2-4cd7-8993-f42907048ae0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb08080f82ff4d73a15c8abd7cb8495f', 'request_id': 'req-cc4f5ada-44a2-4cd7-8993-f42907048ae0'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:41.763483 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cc4f5ada-44a2-4cd7-8993-f42907048ae0 admin admin] The token cache is being invalidate because project fb08080f82ff4d73a15c8abd7cb8495f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:35:41.763873 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cc4f5ada-44a2-4cd7-8993-f42907048ae0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:41.764601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cc4f5ada-44a2-4cd7-8993-f42907048ae0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:41.766240 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1111/2211] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:35:41 2022] DELETE /identity/v3/projects/fb08080f82ff4d73a15c8abd7cb8495f => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:41.771812 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76768c88-e2af-4ce2-9f29-75fbe1320d03 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:41.772372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76768c88-e2af-4ce2-9f29-75fbe1320d03 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:41.772765 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76768c88-e2af-4ce2-9f29-75fbe1320d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:41.792683 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-76768c88-e2af-4ce2-9f29-75fbe1320d03 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:41.793982 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-76768c88-e2af-4ce2-9f29-75fbe1320d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:41.854656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1101/2212] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:35:41.862337 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9400c-f786-4d6f-b06d-e71f086c4777 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:41.862634 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9400c-f786-4d6f-b06d-e71f086c4777 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:41.862918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d9400c-f786-4d6f-b06d-e71f086c4777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:41.883749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-63d9400c-f786-4d6f-b06d-e71f086c4777 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:41.884993 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-63d9400c-f786-4d6f-b06d-e71f086c4777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:41.892738 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1112/2213] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:35:41.896999 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76768c88-e2af-4ce2-9f29-75fbe1320d03 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:41.907551 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76768c88-e2af-4ce2-9f29-75fbe1320d03 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:41.912141 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c6f54-78fc-44ea-8f6d-9656ec8834f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:41.912834 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c6f54-78fc-44ea-8f6d-9656ec8834f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:41.913121 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c6f54-78fc-44ea-8f6d-9656ec8834f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:41.914086 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c6f54-78fc-44ea-8f6d-9656ec8834f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:41.914671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c6f54-78fc-44ea-8f6d-9656ec8834f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:35:41.914976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c6f54-78fc-44ea-8f6d-9656ec8834f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:41.916661 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c6f54-78fc-44ea-8f6d-9656ec8834f7 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:41.929000 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1102/2214] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:35:41.933875 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9400c-f786-4d6f-b06d-e71f086c4777 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:41.945311 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d9400c-f786-4d6f-b06d-e71f086c4777 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:41.950894 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac1dcbc-e85e-4188-b0ef-7904803371c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:41.951748 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1dcbc-e85e-4188-b0ef-7904803371c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:41.952030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1dcbc-e85e-4188-b0ef-7904803371c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:41.952337 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1dcbc-e85e-4188-b0ef-7904803371c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:41.953034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac1dcbc-e85e-4188-b0ef-7904803371c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:41.954928 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac1dcbc-e85e-4188-b0ef-7904803371c1 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:41.993750 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1113/2215] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:35:41 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:35:41.995075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c6f54-78fc-44ea-8f6d-9656ec8834f7 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:42.000450 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:42.006595 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c6f54-78fc-44ea-8f6d-9656ec8834f7 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:42.075725 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f53106-bce0-452d-86da-f1b23d50a7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:42.076704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f53106-bce0-452d-86da-f1b23d50a7eb admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:42.076991 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f53106-bce0-452d-86da-f1b23d50a7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:42.077269 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f53106-bce0-452d-86da-f1b23d50a7eb admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:42.077950 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f53106-bce0-452d-86da-f1b23d50a7eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:42.079868 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f53106-bce0-452d-86da-f1b23d50a7eb admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:42.099892 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1103/2216] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:35:41 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:35:42.104036 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac1dcbc-e85e-4188-b0ef-7904803371c1 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:42.114398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac1dcbc-e85e-4188-b0ef-7904803371c1 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:42.119740 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cec48e-799a-434d-b3a3-53099702a222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:42.120434 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cec48e-799a-434d-b3a3-53099702a222 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:42.120724 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cec48e-799a-434d-b3a3-53099702a222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:42.121006 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cec48e-799a-434d-b3a3-53099702a222 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:42.121731 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cec48e-799a-434d-b3a3-53099702a222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:42.123328 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cec48e-799a-434d-b3a3-53099702a222 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:42.133780 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1114/2217] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:35:42.137594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f53106-bce0-452d-86da-f1b23d50a7eb admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:42.147572 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f53106-bce0-452d-86da-f1b23d50a7eb admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:42.152230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b7d4d-e49e-4a16-86e8-7e19207d5341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:42.152887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b7d4d-e49e-4a16-86e8-7e19207d5341 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:42.153174 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b7d4d-e49e-4a16-86e8-7e19207d5341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:42.153454 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b7d4d-e49e-4a16-86e8-7e19207d5341 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:42.154177 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3b7d4d-e49e-4a16-86e8-7e19207d5341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:42.155780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3b7d4d-e49e-4a16-86e8-7e19207d5341 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:42.166368 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1104/2218] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:35:42.170365 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cec48e-799a-434d-b3a3-53099702a222 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:42.181256 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cec48e-799a-434d-b3a3-53099702a222 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:42.185919 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f1090-a1d0-42f7-9ace-184038644de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:42.186800 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f1090-a1d0-42f7-9ace-184038644de4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:42.187081 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f1090-a1d0-42f7-9ace-184038644de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:42.187354 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f1090-a1d0-42f7-9ace-184038644de4 admin admin] PATH_INFO: `/v3/projects/835f4c15c26e47b7ba55aca85bfb66a8/users/921861792ac246aa8772a636d45489c8/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:42.203058 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f1090-a1d0-42f7-9ace-184038644de4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=835f4c15c26e47b7ba55aca85bfb66a8, user_id=921861792ac246aa8772a636d45489c8, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:42.205676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f1090-a1d0-42f7-9ace-184038644de4 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:42.219656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1115/2219] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:35:42 2022] PUT /identity/v3/projects/835f4c15c26e47b7ba55aca85bfb66a8/users/921861792ac246aa8772a636d45489c8/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:42.224973 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921046b-8561-484a-a294-805316391c6a None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:42.225266 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921046b-8561-484a-a294-805316391c6a None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:42.225568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921046b-8561-484a-a294-805316391c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:42.244983 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-b921046b-8561-484a-a294-805316391c6a None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:42.246199 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-b921046b-8561-484a-a294-805316391c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:42.322862 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1105/2220] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:35:42.332795 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f1090-a1d0-42f7-9ace-184038644de4 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:42.334553 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-f10f1090-a1d0-42f7-9ace-184038644de4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:42.391148 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:42.395373 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f1090-a1d0-42f7-9ace-184038644de4 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:42.402145 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e44774-5f80-459d-8253-d2e406894b1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:42.403020 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e44774-5f80-459d-8253-d2e406894b1a service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:42.403357 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e44774-5f80-459d-8253-d2e406894b1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:42.403910 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e44774-5f80-459d-8253-d2e406894b1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:42.413244 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e44774-5f80-459d-8253-d2e406894b1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:42.415303 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e44774-5f80-459d-8253-d2e406894b1a service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:42.424821 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1116/2221] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:35:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:35:43.999872 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921046b-8561-484a-a294-805316391c6a None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:44.008136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-b921046b-8561-484a-a294-805316391c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:44.175761 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921046b-8561-484a-a294-805316391c6a None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:44.181685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101ff6b-8dd1-4cc1-8806-cf4347d118c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:44.183206 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101ff6b-8dd1-4cc1-8806-cf4347d118c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:44.183936 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101ff6b-8dd1-4cc1-8806-cf4347d118c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:44.184289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101ff6b-8dd1-4cc1-8806-cf4347d118c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:44.186765 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-e101ff6b-8dd1-4cc1-8806-cf4347d118c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:44.253871 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:44.283311 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e101ff6b-8dd1-4cc1-8806-cf4347d118c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:44.284847 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e101ff6b-8dd1-4cc1-8806-cf4347d118c3 service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:44.294862 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1106/2222] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:35:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:35:46.989244 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e44774-5f80-459d-8253-d2e406894b1a service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:46.991365 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-60e44774-5f80-459d-8253-d2e406894b1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:47.027339 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e44774-5f80-459d-8253-d2e406894b1a service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:47.037186 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe107ad-3662-47c4-8086-94d114f82556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:47.038075 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe107ad-3662-47c4-8086-94d114f82556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:47.038969 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe107ad-3662-47c4-8086-94d114f82556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:47.039318 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe107ad-3662-47c4-8086-94d114f82556 admin admin] PATH_INFO: `/v3/users/cb64ee7ac8c54403926403b1e6dfbcad` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:47.044426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe107ad-3662-47c4-8086-94d114f82556 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb64ee7ac8c54403926403b1e6dfbcad)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:47.046501 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe107ad-3662-47c4-8086-94d114f82556 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:47.157289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2fe107ad-3662-47c4-8086-94d114f82556 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb64ee7ac8c54403926403b1e6dfbcad'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:47.217733 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2fe107ad-3662-47c4-8086-94d114f82556 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb64ee7ac8c54403926403b1e6dfbcad'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:47.231286 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2fe107ad-3662-47c4-8086-94d114f82556 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb64ee7ac8c54403926403b1e6dfbcad'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:47.234777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2fe107ad-3662-47c4-8086-94d114f82556 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb64ee7ac8c54403926403b1e6dfbcad'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:47.235933 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2fe107ad-3662-47c4-8086-94d114f82556 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb64ee7ac8c54403926403b1e6dfbcad'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:47.269326 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1117/2223] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:35:46 2022] DELETE /identity/v3/users/cb64ee7ac8c54403926403b1e6dfbcad => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:47.275395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305d362-4afd-4eeb-872d-1aa501df1d25 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:47.276994 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305d362-4afd-4eeb-872d-1aa501df1d25 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:47.278109 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305d362-4afd-4eeb-872d-1aa501df1d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:47.304297 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-f305d362-4afd-4eeb-872d-1aa501df1d25 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:47.305669 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-f305d362-4afd-4eeb-872d-1aa501df1d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:47.386409 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1107/2224] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:35:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:35:47.408558 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe107ad-3662-47c4-8086-94d114f82556 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:47.412235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-2fe107ad-3662-47c4-8086-94d114f82556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:47.498729 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9107d-03aa-4ff7-807b-e53b62aac664 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:47.499355 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9107d-03aa-4ff7-807b-e53b62aac664 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:47.499899 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c9107d-03aa-4ff7-807b-e53b62aac664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:47.549840 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe107ad-3662-47c4-8086-94d114f82556 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:47.558977 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21d44c-0f63-4dc1-8473-2a3e123c19db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:47.559752 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21d44c-0f63-4dc1-8473-2a3e123c19db service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:47.560114 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21d44c-0f63-4dc1-8473-2a3e123c19db service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:47.560487 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21d44c-0f63-4dc1-8473-2a3e123c19db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:47.569021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-51c9107d-03aa-4ff7-807b-e53b62aac664 None None] MFA Rules not processed for user `921861792ac246aa8772a636d45489c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:47.574188 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa21d44c-0f63-4dc1-8473-2a3e123c19db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:47.576047 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa21d44c-0f63-4dc1-8473-2a3e123c19db service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:47.579063 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-51c9107d-03aa-4ff7-807b-e53b62aac664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:47.589981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1118/2225] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:35:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:35:47.705327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1108/2226] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:35:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:35:47.709980 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21d44c-0f63-4dc1-8473-2a3e123c19db service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:47.712886 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-fa21d44c-0f63-4dc1-8473-2a3e123c19db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:47.736138 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21d44c-0f63-4dc1-8473-2a3e123c19db service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:47.743467 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913cf4dc-3c62-4f2c-ae13-b9ab9a8c1ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:47.744360 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913cf4dc-3c62-4f2c-ae13-b9ab9a8c1ceb admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:47.745223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913cf4dc-3c62-4f2c-ae13-b9ab9a8c1ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:47.745316 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913cf4dc-3c62-4f2c-ae13-b9ab9a8c1ceb admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:47.746189 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913cf4dc-3c62-4f2c-ae13-b9ab9a8c1ceb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:47.748804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913cf4dc-3c62-4f2c-ae13-b9ab9a8c1ceb admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:47.791287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1119/2227] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:35:47 2022] POST /identity/v3/projects => generated 371 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:35:47.796207 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c9107d-03aa-4ff7-807b-e53b62aac664 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:47.821229 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c9107d-03aa-4ff7-807b-e53b62aac664 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:47.908676 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e0fbf-43b2-4331-b26b-5cd2e7178bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:47.909357 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e0fbf-43b2-4331-b26b-5cd2e7178bd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:47.909648 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e0fbf-43b2-4331-b26b-5cd2e7178bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:47.909928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e0fbf-43b2-4331-b26b-5cd2e7178bd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:47.910719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20e0fbf-43b2-4331-b26b-5cd2e7178bd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:47.913042 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20e0fbf-43b2-4331-b26b-5cd2e7178bd7 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:47.925332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:47.941699 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1109/2228] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:35:47 2022] POST /identity/v3/users => generated 292 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:35:47.945560 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913cf4dc-3c62-4f2c-ae13-b9ab9a8c1ceb admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:47.958415 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913cf4dc-3c62-4f2c-ae13-b9ab9a8c1ceb admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:47.963348 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c1930-2850-4128-a814-c87ee35e668f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:47.964099 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c1930-2850-4128-a814-c87ee35e668f admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:47.964503 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c1930-2850-4128-a814-c87ee35e668f admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:47.964967 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c1930-2850-4128-a814-c87ee35e668f admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:47.965768 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c1930-2850-4128-a814-c87ee35e668f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:47.967451 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c1930-2850-4128-a814-c87ee35e668f admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:47.983383 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1120/2229] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:35:47.984760 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e0fbf-43b2-4331-b26b-5cd2e7178bd7 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:47.996088 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e0fbf-43b2-4331-b26b-5cd2e7178bd7 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:48.003344 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d32ab3-fb2e-4807-9f97-07235750874d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:48.004279 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d32ab3-fb2e-4807-9f97-07235750874d admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:48.004594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d32ab3-fb2e-4807-9f97-07235750874d admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:48.005299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d32ab3-fb2e-4807-9f97-07235750874d admin admin] PATH_INFO: `/v3/projects/48239413b5fe44c3905c37d9a647a41b/users/d787695b76ed4b09b744b3ac69fcbe4d/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:48.025431 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d32ab3-fb2e-4807-9f97-07235750874d admin admin] RBAC: Authorizing `identity:create_grant(project_id=48239413b5fe44c3905c37d9a647a41b, user_id=d787695b76ed4b09b744b3ac69fcbe4d, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:48.028352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d32ab3-fb2e-4807-9f97-07235750874d admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:48.042996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1110/2230] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:35:47 2022] PUT /identity/v3/projects/48239413b5fe44c3905c37d9a647a41b/users/d787695b76ed4b09b744b3ac69fcbe4d/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:48.047115 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c1930-2850-4128-a814-c87ee35e668f admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:48.058766 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c1930-2850-4128-a814-c87ee35e668f admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:48.100251 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:48.162247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054c3bf4-4143-40a0-8ae3-d77e83542ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:48.162997 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c3bf4-4143-40a0-8ae3-d77e83542ebf admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:48.163480 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c3bf4-4143-40a0-8ae3-d77e83542ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:48.164431 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c3bf4-4143-40a0-8ae3-d77e83542ebf admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:48.165326 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054c3bf4-4143-40a0-8ae3-d77e83542ebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:48.166916 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054c3bf4-4143-40a0-8ae3-d77e83542ebf admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:48.182817 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d32ab3-fb2e-4807-9f97-07235750874d admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:48.184736 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-30d32ab3-fb2e-4807-9f97-07235750874d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:48.192436 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1121/2231] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:35:48.195575 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054c3bf4-4143-40a0-8ae3-d77e83542ebf admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:48.204832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d32ab3-fb2e-4807-9f97-07235750874d admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:48.220512 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054c3bf4-4143-40a0-8ae3-d77e83542ebf admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:48.225567 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9a3ed-1ddc-44b2-bfdc-f5e14fe3811a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:48.228197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9a3ed-1ddc-44b2-bfdc-f5e14fe3811a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:48.228811 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9a3ed-1ddc-44b2-bfdc-f5e14fe3811a admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:48.229174 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9a3ed-1ddc-44b2-bfdc-f5e14fe3811a admin admin] PATH_INFO: `/v3/projects/992fa1b6160f48f59ae18307256b6f72` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:48.230298 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d9a3ed-1ddc-44b2-bfdc-f5e14fe3811a admin admin] RBAC: Authorizing `identity:delete_project(project_id=992fa1b6160f48f59ae18307256b6f72)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:48.232279 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d9a3ed-1ddc-44b2-bfdc-f5e14fe3811a admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:48.253261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2e752-5613-452a-8015-b4874861916a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:48.257136 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2e752-5613-452a-8015-b4874861916a admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:48.257427 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2e752-5613-452a-8015-b4874861916a admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:48.257711 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2e752-5613-452a-8015-b4874861916a admin admin] PATH_INFO: `/v3/projects/48239413b5fe44c3905c37d9a647a41b/users/d787695b76ed4b09b744b3ac69fcbe4d/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:48.264566 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e2e752-5613-452a-8015-b4874861916a admin admin] RBAC: Authorizing `identity:create_grant(project_id=48239413b5fe44c3905c37d9a647a41b, user_id=d787695b76ed4b09b744b3ac69fcbe4d, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:48.269398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e2e752-5613-452a-8015-b4874861916a admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:48.301632 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1122/2232] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:35:48 2022] PUT /identity/v3/projects/48239413b5fe44c3905c37d9a647a41b/users/d787695b76ed4b09b744b3ac69fcbe4d/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:48.363079 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-45d9a3ed-1ddc-44b2-bfdc-f5e14fe3811a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '992fa1b6160f48f59ae18307256b6f72', 'request_id': 'req-45d9a3ed-1ddc-44b2-bfdc-f5e14fe3811a'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:48.381518 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-45d9a3ed-1ddc-44b2-bfdc-f5e14fe3811a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '992fa1b6160f48f59ae18307256b6f72', 'request_id': 'req-45d9a3ed-1ddc-44b2-bfdc-f5e14fe3811a'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:48.421016 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-45d9a3ed-1ddc-44b2-bfdc-f5e14fe3811a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '992fa1b6160f48f59ae18307256b6f72', 'request_id': 'req-45d9a3ed-1ddc-44b2-bfdc-f5e14fe3811a'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:48.461470 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-45d9a3ed-1ddc-44b2-bfdc-f5e14fe3811a admin admin] The token cache is being invalidate because project 992fa1b6160f48f59ae18307256b6f72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:35:48.461967 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-45d9a3ed-1ddc-44b2-bfdc-f5e14fe3811a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:48.462798 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-45d9a3ed-1ddc-44b2-bfdc-f5e14fe3811a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:48.468012 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1111/2233] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:35:48 2022] DELETE /identity/v3/projects/992fa1b6160f48f59ae18307256b6f72 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:48.469398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2e752-5613-452a-8015-b4874861916a admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:48.472740 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-06e2e752-5613-452a-8015-b4874861916a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:48.680495 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2e752-5613-452a-8015-b4874861916a admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:48.779472 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45c62f-3f76-4952-83b6-0bb13cfb009e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:48.780494 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45c62f-3f76-4952-83b6-0bb13cfb009e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:48.780815 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45c62f-3f76-4952-83b6-0bb13cfb009e admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:48.781307 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45c62f-3f76-4952-83b6-0bb13cfb009e admin admin] PATH_INFO: `/v3/users/582ef8464e914e379d913747ae2d1e17` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:48.783873 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb45c62f-3f76-4952-83b6-0bb13cfb009e admin admin] RBAC: Authorizing `identity:delete_user(user_id=582ef8464e914e379d913747ae2d1e17)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:48.786421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb45c62f-3f76-4952-83b6-0bb13cfb009e admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:48.883878 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-bb45c62f-3f76-4952-83b6-0bb13cfb009e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '582ef8464e914e379d913747ae2d1e17'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:48.906706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-bb45c62f-3f76-4952-83b6-0bb13cfb009e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '582ef8464e914e379d913747ae2d1e17'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:48.911420 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:48.926271 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-bb45c62f-3f76-4952-83b6-0bb13cfb009e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '582ef8464e914e379d913747ae2d1e17'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:48.927291 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-bb45c62f-3f76-4952-83b6-0bb13cfb009e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '582ef8464e914e379d913747ae2d1e17'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:48.929625 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-bb45c62f-3f76-4952-83b6-0bb13cfb009e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '582ef8464e914e379d913747ae2d1e17'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:48.987965 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1123/2234] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:35:48 2022] DELETE /identity/v3/users/582ef8464e914e379d913747ae2d1e17 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:49.433515 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9a3ed-1ddc-44b2-bfdc-f5e14fe3811a admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:49.456769 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9a3ed-1ddc-44b2-bfdc-f5e14fe3811a admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:49.543648 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81369e9-5084-4ca8-ae87-940996b9b084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:49.544898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81369e9-5084-4ca8-ae87-940996b9b084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:49.545282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81369e9-5084-4ca8-ae87-940996b9b084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:49.545802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81369e9-5084-4ca8-ae87-940996b9b084 admin admin] PATH_INFO: `/v3/projects/d0be3eb87853483a9e214f43cc520b7b` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:49.547744 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81369e9-5084-4ca8-ae87-940996b9b084 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0be3eb87853483a9e214f43cc520b7b)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:49.551061 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81369e9-5084-4ca8-ae87-940996b9b084 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:49.631459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f81369e9-5084-4ca8-ae87-940996b9b084 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0be3eb87853483a9e214f43cc520b7b', 'request_id': 'req-f81369e9-5084-4ca8-ae87-940996b9b084'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:49.642451 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f81369e9-5084-4ca8-ae87-940996b9b084 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0be3eb87853483a9e214f43cc520b7b', 'request_id': 'req-f81369e9-5084-4ca8-ae87-940996b9b084'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:49.643982 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=85,keystone:INSERT=3,keystone:DELETE=12 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:49.653530 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f81369e9-5084-4ca8-ae87-940996b9b084 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0be3eb87853483a9e214f43cc520b7b', 'request_id': 'req-f81369e9-5084-4ca8-ae87-940996b9b084'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:49.669747 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f81369e9-5084-4ca8-ae87-940996b9b084 admin admin] The token cache is being invalidate because project d0be3eb87853483a9e214f43cc520b7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:35:49.671117 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f81369e9-5084-4ca8-ae87-940996b9b084 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:49.671117 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f81369e9-5084-4ca8-ae87-940996b9b084 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:49.673439 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1112/2235] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:35:49 2022] DELETE /identity/v3/projects/d0be3eb87853483a9e214f43cc520b7b => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:49.677204 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45c62f-3f76-4952-83b6-0bb13cfb009e admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:49.678808 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-bb45c62f-3f76-4952-83b6-0bb13cfb009e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:49.693962 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45c62f-3f76-4952-83b6-0bb13cfb009e admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:49.772274 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2d74f-8509-4002-bda0-cc9db173083a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:49.774595 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2d74f-8509-4002-bda0-cc9db173083a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:49.774780 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2d74f-8509-4002-bda0-cc9db173083a admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:49.775050 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2d74f-8509-4002-bda0-cc9db173083a admin admin] PATH_INFO: `/v3/users/2730479ac8374af7ac435c6ef5a90dd6` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:49.776318 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b2d74f-8509-4002-bda0-cc9db173083a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2730479ac8374af7ac435c6ef5a90dd6)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:49.778271 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b2d74f-8509-4002-bda0-cc9db173083a admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:49.852855 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-65b2d74f-8509-4002-bda0-cc9db173083a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2730479ac8374af7ac435c6ef5a90dd6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:49.876423 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-65b2d74f-8509-4002-bda0-cc9db173083a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2730479ac8374af7ac435c6ef5a90dd6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:49.889341 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-65b2d74f-8509-4002-bda0-cc9db173083a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2730479ac8374af7ac435c6ef5a90dd6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:49.890839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-65b2d74f-8509-4002-bda0-cc9db173083a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2730479ac8374af7ac435c6ef5a90dd6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:49.892481 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-65b2d74f-8509-4002-bda0-cc9db173083a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2730479ac8374af7ac435c6ef5a90dd6'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:49.910258 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1124/2236] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:35:49 2022] DELETE /identity/v3/users/2730479ac8374af7ac435c6ef5a90dd6 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:35:50.634084 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81369e9-5084-4ca8-ae87-940996b9b084 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:50.635592 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-f81369e9-5084-4ca8-ae87-940996b9b084 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:50.658120 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81369e9-5084-4ca8-ae87-940996b9b084 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:50.783241 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0098e87e-e18d-49a5-a54d-1b0b2f0a9e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:50.784420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098e87e-e18d-49a5-a54d-1b0b2f0a9e4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:50.784871 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098e87e-e18d-49a5-a54d-1b0b2f0a9e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:50.785334 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098e87e-e18d-49a5-a54d-1b0b2f0a9e4e admin admin] PATH_INFO: `/v3/projects/f8b67aa9f1934f4a9612bd3c419fab57` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:50.786777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0098e87e-e18d-49a5-a54d-1b0b2f0a9e4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8b67aa9f1934f4a9612bd3c419fab57)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:50.790691 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0098e87e-e18d-49a5-a54d-1b0b2f0a9e4e admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:50.880923 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-0098e87e-e18d-49a5-a54d-1b0b2f0a9e4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8b67aa9f1934f4a9612bd3c419fab57', 'request_id': 'req-0098e87e-e18d-49a5-a54d-1b0b2f0a9e4e'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:50.894349 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-0098e87e-e18d-49a5-a54d-1b0b2f0a9e4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8b67aa9f1934f4a9612bd3c419fab57', 'request_id': 'req-0098e87e-e18d-49a5-a54d-1b0b2f0a9e4e'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:50.906638 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-0098e87e-e18d-49a5-a54d-1b0b2f0a9e4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8b67aa9f1934f4a9612bd3c419fab57', 'request_id': 'req-0098e87e-e18d-49a5-a54d-1b0b2f0a9e4e'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:50.921178 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-0098e87e-e18d-49a5-a54d-1b0b2f0a9e4e admin admin] The token cache is being invalidate because project f8b67aa9f1934f4a9612bd3c419fab57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:35:50.921579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-0098e87e-e18d-49a5-a54d-1b0b2f0a9e4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:50.922405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-0098e87e-e18d-49a5-a54d-1b0b2f0a9e4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:50.925146 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1113/2237] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:35:50 2022] DELETE /identity/v3/projects/f8b67aa9f1934f4a9612bd3c419fab57 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:35:50.931663 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b7221-4a45-4164-9acd-5e013efd35a8 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:50.932221 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b7221-4a45-4164-9acd-5e013efd35a8 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:50.932341 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b7221-4a45-4164-9acd-5e013efd35a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:50.968263 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-865b7221-4a45-4164-9acd-5e013efd35a8 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:50.969509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-865b7221-4a45-4164-9acd-5e013efd35a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:51.088986 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1125/2238] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:35:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:35:51.090867 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b782227-a3dd-449a-b551-167f9e584663 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:51.091185 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b782227-a3dd-449a-b551-167f9e584663 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:51.091581 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b782227-a3dd-449a-b551-167f9e584663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:51.131681 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-1b782227-a3dd-449a-b551-167f9e584663 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:51.135697 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-1b782227-a3dd-449a-b551-167f9e584663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:51.156257 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1114/2239] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:35:51.157826 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b7221-4a45-4164-9acd-5e013efd35a8 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:51.187534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:51.197297 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b7221-4a45-4164-9acd-5e013efd35a8 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:51.209561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ffead1-e4da-4961-a737-d27fbe94bdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:51.213156 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ffead1-e4da-4961-a737-d27fbe94bdcd admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:51.213455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ffead1-e4da-4961-a737-d27fbe94bdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:51.213743 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ffead1-e4da-4961-a737-d27fbe94bdcd admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:51.214547 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ffead1-e4da-4961-a737-d27fbe94bdcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:35:51.214866 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ffead1-e4da-4961-a737-d27fbe94bdcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:51.220632 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ffead1-e4da-4961-a737-d27fbe94bdcd admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:51.235934 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1126/2240] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:35:51.240149 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b782227-a3dd-449a-b551-167f9e584663 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:51.258716 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b782227-a3dd-449a-b551-167f9e584663 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:51.273407 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140cc12b-5626-4757-8277-014e1d133547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:51.274412 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cc12b-5626-4757-8277-014e1d133547 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:51.274794 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cc12b-5626-4757-8277-014e1d133547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:51.275174 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140cc12b-5626-4757-8277-014e1d133547 admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:51.276002 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140cc12b-5626-4757-8277-014e1d133547 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:51.278066 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140cc12b-5626-4757-8277-014e1d133547 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:51.322204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1115/2241] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:35:51 2022] POST /identity/v3/projects => generated 373 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:35:51.323518 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ffead1-e4da-4961-a737-d27fbe94bdcd admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:51.337880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ffead1-e4da-4961-a737-d27fbe94bdcd admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:51.411755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e5f1f-7032-4b99-aeb7-08d33463b448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:51.413974 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e5f1f-7032-4b99-aeb7-08d33463b448 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:51.414422 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e5f1f-7032-4b99-aeb7-08d33463b448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:51.414807 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e5f1f-7032-4b99-aeb7-08d33463b448 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:51.415603 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e5f1f-7032-4b99-aeb7-08d33463b448 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:51.418835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e5f1f-7032-4b99-aeb7-08d33463b448 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:51.448048 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1127/2242] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:35:51 2022] POST /identity/v3/users => generated 287 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:35:51.452862 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140cc12b-5626-4757-8277-014e1d133547 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:51.466115 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140cc12b-5626-4757-8277-014e1d133547 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:51.471248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ff425-a95a-41c4-9bf2-eaff03ef0d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:51.472019 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ff425-a95a-41c4-9bf2-eaff03ef0d55 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:51.472545 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ff425-a95a-41c4-9bf2-eaff03ef0d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:51.473022 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ff425-a95a-41c4-9bf2-eaff03ef0d55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:51.473861 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ff425-a95a-41c4-9bf2-eaff03ef0d55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:51.475703 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896ff425-a95a-41c4-9bf2-eaff03ef0d55 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:51.489120 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1116/2243] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:35:51.495222 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e5f1f-7032-4b99-aeb7-08d33463b448 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:51.529367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e5f1f-7032-4b99-aeb7-08d33463b448 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:51.548091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6143175-ea2b-467e-b7e9-662fb959a7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:51.548819 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6143175-ea2b-467e-b7e9-662fb959a7e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:51.549111 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6143175-ea2b-467e-b7e9-662fb959a7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:51.549377 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6143175-ea2b-467e-b7e9-662fb959a7e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:51.550115 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6143175-ea2b-467e-b7e9-662fb959a7e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:51.551710 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6143175-ea2b-467e-b7e9-662fb959a7e9 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:51.588187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ff425-a95a-41c4-9bf2-eaff03ef0d55 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:51.598258 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1128/2244] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:35:51.601751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ff425-a95a-41c4-9bf2-eaff03ef0d55 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:51.620806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73fcff2-75da-4d3e-936c-6408fb914e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:51.621636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73fcff2-75da-4d3e-936c-6408fb914e78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:51.621922 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73fcff2-75da-4d3e-936c-6408fb914e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:51.622199 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73fcff2-75da-4d3e-936c-6408fb914e78 admin admin] PATH_INFO: `/v3/projects/4a961cf77299480e8f3dc1b1e32fe7a0/users/6a4e8e30b7c742ca969c685a92807adb/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:51.654497 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73fcff2-75da-4d3e-936c-6408fb914e78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a961cf77299480e8f3dc1b1e32fe7a0, user_id=6a4e8e30b7c742ca969c685a92807adb, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:51.657692 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73fcff2-75da-4d3e-936c-6408fb914e78 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:51.700705 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1117/2245] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:35:51 2022] PUT /identity/v3/projects/4a961cf77299480e8f3dc1b1e32fe7a0/users/6a4e8e30b7c742ca969c685a92807adb/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:35:51.715562 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de88281-0823-40fa-8047-125484138346 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:51.715858 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de88281-0823-40fa-8047-125484138346 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:51.716138 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de88281-0823-40fa-8047-125484138346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:51.796146 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-9de88281-0823-40fa-8047-125484138346 None None] MFA Rules not processed for user `6a4e8e30b7c742ca969c685a92807adb`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:51.799597 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-9de88281-0823-40fa-8047-125484138346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:51.909442 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1129/2246] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:35:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:35:51.915098 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73fcff2-75da-4d3e-936c-6408fb914e78 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:51.938432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:52.093553 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73fcff2-75da-4d3e-936c-6408fb914e78 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:52.108910 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b063a2-2a49-432b-9d2f-597e087ece50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:52.109929 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b063a2-2a49-432b-9d2f-597e087ece50 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:52.110086 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b063a2-2a49-432b-9d2f-597e087ece50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:52.110532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b063a2-2a49-432b-9d2f-597e087ece50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:52.111368 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b063a2-2a49-432b-9d2f-597e087ece50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:52.118238 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b063a2-2a49-432b-9d2f-597e087ece50 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:52.174538 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de88281-0823-40fa-8047-125484138346 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:52.189211 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1118/2247] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:35:51 2022] POST /identity/v3/projects => generated 375 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:35:52.200626 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de88281-0823-40fa-8047-125484138346 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:52.260754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c550374-f2c1-425f-acfb-215e0df8307c None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:52.261372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c550374-f2c1-425f-acfb-215e0df8307c None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:52.261837 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c550374-f2c1-425f-acfb-215e0df8307c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:52.312593 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:52.321500 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b786a-62be-41d2-9cb3-b5d18db48c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:52.323690 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b786a-62be-41d2-9cb3-b5d18db48c77 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:52.325900 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b786a-62be-41d2-9cb3-b5d18db48c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:52.325900 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b786a-62be-41d2-9cb3-b5d18db48c77 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:52.327259 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7b786a-62be-41d2-9cb3-b5d18db48c77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:52.332481 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7b786a-62be-41d2-9cb3-b5d18db48c77 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:52.400415 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1130/2248] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:35:52 2022] POST /identity/v3/users => generated 294 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:35:52.409624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b786a-62be-41d2-9cb3-b5d18db48c77 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:52.442138 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b786a-62be-41d2-9cb3-b5d18db48c77 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:52.445078 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-5c550374-f2c1-425f-acfb-215e0df8307c None None] MFA Rules not processed for user `733f8296ec3d4795a1140c1e1cb6eb95`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:52.447129 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-5c550374-f2c1-425f-acfb-215e0df8307c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:52.455098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d33af3e-e879-4719-8007-d5bef9a242d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:52.455547 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33af3e-e879-4719-8007-d5bef9a242d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:52.455807 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33af3e-e879-4719-8007-d5bef9a242d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:52.456089 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33af3e-e879-4719-8007-d5bef9a242d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:52.456904 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d33af3e-e879-4719-8007-d5bef9a242d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:52.462574 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d33af3e-e879-4719-8007-d5bef9a242d7 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:52.487712 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1131/2249] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:35:52.488914 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d33af3e-e879-4719-8007-d5bef9a242d7 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:52.526900 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d33af3e-e879-4719-8007-d5bef9a242d7 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:52.541041 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dda94c8-9a4b-4547-b0a4-e9955ada3241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:52.541973 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda94c8-9a4b-4547-b0a4-e9955ada3241 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:52.542275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda94c8-9a4b-4547-b0a4-e9955ada3241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:52.542541 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dda94c8-9a4b-4547-b0a4-e9955ada3241 admin admin] PATH_INFO: `/v3/projects/9560bf66ad094c3fb3e17783dd59e504/users/b57c5d470ef1490dbe62ea0c3dcdd061/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:52.568751 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dda94c8-9a4b-4547-b0a4-e9955ada3241 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9560bf66ad094c3fb3e17783dd59e504, user_id=b57c5d470ef1490dbe62ea0c3dcdd061, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:52.572223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dda94c8-9a4b-4547-b0a4-e9955ada3241 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:52.593298 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1119/2250] 10.222.0.16 () {58 vars in 1009 bytes} [Mon May 23 12:35:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:35:52.608523 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1132/2251] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:35:52 2022] PUT /identity/v3/projects/9560bf66ad094c3fb3e17783dd59e504/users/b57c5d470ef1490dbe62ea0c3dcdd061/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:35:52.612682 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c550374-f2c1-425f-acfb-215e0df8307c None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:52.618112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dda94c8-9a4b-4547-b0a4-e9955ada3241 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:52.621131 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-7dda94c8-9a4b-4547-b0a4-e9955ada3241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:52.628275 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c550374-f2c1-425f-acfb-215e0df8307c None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:52.726410 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffe70b-2b46-4735-8f80-65ecac01d561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:52.727603 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffe70b-2b46-4735-8f80-65ecac01d561 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:52.727907 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffe70b-2b46-4735-8f80-65ecac01d561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:52.728189 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ffe70b-2b46-4735-8f80-65ecac01d561 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:52.728956 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ffe70b-2b46-4735-8f80-65ecac01d561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:52.730590 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ffe70b-2b46-4735-8f80-65ecac01d561 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:52.742731 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:52.802242 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1120/2252] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:35:52.803215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffe70b-2b46-4735-8f80-65ecac01d561 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:52.818187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dda94c8-9a4b-4547-b0a4-e9955ada3241 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:52.822592 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ffe70b-2b46-4735-8f80-65ecac01d561 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:52.830333 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d135e5ea-eab2-4fac-a014-b843d893a50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:52.831256 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135e5ea-eab2-4fac-a014-b843d893a50f admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:52.831504 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135e5ea-eab2-4fac-a014-b843d893a50f admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:52.831787 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135e5ea-eab2-4fac-a014-b843d893a50f admin admin] PATH_INFO: `/v3/projects/9560bf66ad094c3fb3e17783dd59e504/users/b57c5d470ef1490dbe62ea0c3dcdd061/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:52.833971 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d135e5ea-eab2-4fac-a014-b843d893a50f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9560bf66ad094c3fb3e17783dd59e504, user_id=b57c5d470ef1490dbe62ea0c3dcdd061, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:52.836701 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340fe8f-742e-4574-9127-d2210b2bda37 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71fb9a79ac494635804b93d4a0048205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:52.838959 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340fe8f-742e-4574-9127-d2210b2bda37 service swift] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:52.839430 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340fe8f-742e-4574-9127-d2210b2bda37 service swift] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:52.839643 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d135e5ea-eab2-4fac-a014-b843d893a50f admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:52.840091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340fe8f-742e-4574-9127-d2210b2bda37 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:52.871275 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1121/2253] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:35:52 2022] PUT /identity/v3/projects/9560bf66ad094c3fb3e17783dd59e504/users/b57c5d470ef1490dbe62ea0c3dcdd061/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:52.873326 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92bedcf-0332-4de7-adee-0617add55126 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:52.873656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92bedcf-0332-4de7-adee-0617add55126 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:52.873939 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92bedcf-0332-4de7-adee-0617add55126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:52.917299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-c92bedcf-0332-4de7-adee-0617add55126 None None] MFA Rules not processed for user `b57c5d470ef1490dbe62ea0c3dcdd061`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:52.918606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-c92bedcf-0332-4de7-adee-0617add55126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:53.012922 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2340fe8f-742e-4574-9127-d2210b2bda37 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:53.017249 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2340fe8f-742e-4574-9127-d2210b2bda37 service swift] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:53.045630 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1122/2254] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:35:52 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:35:53.056359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92bedcf-0332-4de7-adee-0617add55126 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:53.060100 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-c92bedcf-0332-4de7-adee-0617add55126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:53.064723 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1133/2255] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 12:35:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 23 12:35:53.284314 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92bedcf-0332-4de7-adee-0617add55126 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:53.297330 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ac762-c48c-4fc5-b93c-edde8325f6c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:53.298217 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ac762-c48c-4fc5-b93c-edde8325f6c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:53.299549 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ac762-c48c-4fc5-b93c-edde8325f6c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:53.300214 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ac762-c48c-4fc5-b93c-edde8325f6c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:53.520635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3ac762-c48c-4fc5-b93c-edde8325f6c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:53.522120 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3ac762-c48c-4fc5-b93c-edde8325f6c5 service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:53.562887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1123/2256] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:35:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:35:56.926932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340fe8f-742e-4574-9127-d2210b2bda37 service swift] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:56.962125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340fe8f-742e-4574-9127-d2210b2bda37 service swift] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:57.049269 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:57.113436 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ee3ff-3870-4b5c-bb00-53936dbbbe46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:57.114612 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ee3ff-3870-4b5c-bb00-53936dbbbe46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:57.120842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ee3ff-3870-4b5c-bb00-53936dbbbe46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:57.121361 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ee3ff-3870-4b5c-bb00-53936dbbbe46 admin admin] PATH_INFO: `/v3/users/6a4e8e30b7c742ca969c685a92807adb` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:57.123446 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ee3ff-3870-4b5c-bb00-53936dbbbe46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a4e8e30b7c742ca969c685a92807adb)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:57.125731 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ee3ff-3870-4b5c-bb00-53936dbbbe46 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:57.243129 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-542ee3ff-3870-4b5c-bb00-53936dbbbe46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a4e8e30b7c742ca969c685a92807adb'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:57.272250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-542ee3ff-3870-4b5c-bb00-53936dbbbe46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a4e8e30b7c742ca969c685a92807adb'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:57.299741 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-542ee3ff-3870-4b5c-bb00-53936dbbbe46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a4e8e30b7c742ca969c685a92807adb'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:57.304248 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-542ee3ff-3870-4b5c-bb00-53936dbbbe46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a4e8e30b7c742ca969c685a92807adb'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:57.305378 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-542ee3ff-3870-4b5c-bb00-53936dbbbe46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a4e8e30b7c742ca969c685a92807adb'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:57.326967 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1134/2257] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:35:56 2022] DELETE /identity/v3/users/6a4e8e30b7c742ca969c685a92807adb => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:57.332685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24571c5-2891-480a-85c0-3613cd1cbca2 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:57.336991 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24571c5-2891-480a-85c0-3613cd1cbca2 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:57.338437 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24571c5-2891-480a-85c0-3613cd1cbca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:57.404345 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-d24571c5-2891-480a-85c0-3613cd1cbca2 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:57.405665 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-d24571c5-2891-480a-85c0-3613cd1cbca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:57.514815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:57.631153 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1124/2258] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:35:57.715083 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ee3ff-3870-4b5c-bb00-53936dbbbe46 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:57.716952 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-542ee3ff-3870-4b5c-bb00-53936dbbbe46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:57.740534 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad9838-5e5c-4162-b6ab-c6d1b6e0ba3a None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:57.741167 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad9838-5e5c-4162-b6ab-c6d1b6e0ba3a None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:57.741685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad9838-5e5c-4162-b6ab-c6d1b6e0ba3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:57.831088 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-c4ad9838-5e5c-4162-b6ab-c6d1b6e0ba3a None None] MFA Rules not processed for user `d787695b76ed4b09b744b3ac69fcbe4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:35:57.833374 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-c4ad9838-5e5c-4162-b6ab-c6d1b6e0ba3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:57.894730 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ee3ff-3870-4b5c-bb00-53936dbbbe46 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:57.902132 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a54c0-0b55-44d8-95b6-dff0817b812f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:57.902953 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a54c0-0b55-44d8-95b6-dff0817b812f service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:57.903353 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a54c0-0b55-44d8-95b6-dff0817b812f service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:57.903752 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a54c0-0b55-44d8-95b6-dff0817b812f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:57.931687 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25a54c0-0b55-44d8-95b6-dff0817b812f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:57.933427 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25a54c0-0b55-44d8-95b6-dff0817b812f service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:57.949529 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1135/2259] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:35:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:35:58.087102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1125/2260] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:35:57 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:35:58.106757 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a54c0-0b55-44d8-95b6-dff0817b812f service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:58.110372 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e25a54c0-0b55-44d8-95b6-dff0817b812f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:58.203932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:58.256199 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a54c0-0b55-44d8-95b6-dff0817b812f service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:58.263897 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff8fff-b6c2-44ed-ad66-fa62e8f3a685 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:58.264871 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff8fff-b6c2-44ed-ad66-fa62e8f3a685 service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:58.266111 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff8fff-b6c2-44ed-ad66-fa62e8f3a685 service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:58.266517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff8fff-b6c2-44ed-ad66-fa62e8f3a685 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:58.269037 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a4ff8fff-b6c2-44ed-ad66-fa62e8f3a685 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:58.394239 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ff8fff-b6c2-44ed-ad66-fa62e8f3a685 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:58.396821 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ff8fff-b6c2-44ed-ad66-fa62e8f3a685 service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:58.435120 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1136/2261] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:35:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:35:58.683706 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad9838-5e5c-4162-b6ab-c6d1b6e0ba3a None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:58.685491 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-c4ad9838-5e5c-4162-b6ab-c6d1b6e0ba3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:35:58.737855 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad9838-5e5c-4162-b6ab-c6d1b6e0ba3a None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:58.752204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71a52d-0663-4189-8c0a-2c17c68e32d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:58.754137 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71a52d-0663-4189-8c0a-2c17c68e32d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:58.754715 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71a52d-0663-4189-8c0a-2c17c68e32d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:58.755745 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71a52d-0663-4189-8c0a-2c17c68e32d8 admin admin] PATH_INFO: `/v3/projects/4a961cf77299480e8f3dc1b1e32fe7a0` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:58.758442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf71a52d-0663-4189-8c0a-2c17c68e32d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a961cf77299480e8f3dc1b1e32fe7a0)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:58.761564 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf71a52d-0663-4189-8c0a-2c17c68e32d8 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:58.890707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-cf71a52d-0663-4189-8c0a-2c17c68e32d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a961cf77299480e8f3dc1b1e32fe7a0', 'request_id': 'req-cf71a52d-0663-4189-8c0a-2c17c68e32d8'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:58.919756 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-cf71a52d-0663-4189-8c0a-2c17c68e32d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a961cf77299480e8f3dc1b1e32fe7a0', 'request_id': 'req-cf71a52d-0663-4189-8c0a-2c17c68e32d8'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:58.934227 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-cf71a52d-0663-4189-8c0a-2c17c68e32d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a961cf77299480e8f3dc1b1e32fe7a0', 'request_id': 'req-cf71a52d-0663-4189-8c0a-2c17c68e32d8'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:58.975071 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-cf71a52d-0663-4189-8c0a-2c17c68e32d8 admin admin] The token cache is being invalidate because project 4a961cf77299480e8f3dc1b1e32fe7a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:35:58.975493 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-cf71a52d-0663-4189-8c0a-2c17c68e32d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:58.976967 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-cf71a52d-0663-4189-8c0a-2c17c68e32d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:58.979701 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1126/2262] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:35:58 2022] DELETE /identity/v3/projects/4a961cf77299480e8f3dc1b1e32fe7a0 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:35:58.982916 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff8fff-b6c2-44ed-ad66-fa62e8f3a685 service nova] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:35:59.000368 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff8fff-b6c2-44ed-ad66-fa62e8f3a685 service nova] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:35:59.169402 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44882727-43f2-4433-9231-4ced13ca96c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:35:59.172871 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44882727-43f2-4433-9231-4ced13ca96c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:35:59.173280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44882727-43f2-4433-9231-4ced13ca96c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:35:59.173670 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44882727-43f2-4433-9231-4ced13ca96c2 admin admin] PATH_INFO: `/v3/users/b57c5d470ef1490dbe62ea0c3dcdd061` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:35:59.177253 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44882727-43f2-4433-9231-4ced13ca96c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b57c5d470ef1490dbe62ea0c3dcdd061)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:35:59.182117 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44882727-43f2-4433-9231-4ced13ca96c2 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:35:59.313171 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-44882727-43f2-4433-9231-4ced13ca96c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b57c5d470ef1490dbe62ea0c3dcdd061'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:59.349865 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-44882727-43f2-4433-9231-4ced13ca96c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b57c5d470ef1490dbe62ea0c3dcdd061'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:59.380290 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-44882727-43f2-4433-9231-4ced13ca96c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b57c5d470ef1490dbe62ea0c3dcdd061'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:59.381244 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-44882727-43f2-4433-9231-4ced13ca96c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b57c5d470ef1490dbe62ea0c3dcdd061'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:59.382459 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-44882727-43f2-4433-9231-4ced13ca96c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b57c5d470ef1490dbe62ea0c3dcdd061'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:35:59.385857 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:35:59.421843 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1137/2263] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:35:58 2022] DELETE /identity/v3/users/b57c5d470ef1490dbe62ea0c3dcdd061 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:36:00.072463 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71a52d-0663-4189-8c0a-2c17c68e32d8 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:36:00.073977 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-cf71a52d-0663-4189-8c0a-2c17c68e32d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:36:00.097111 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71a52d-0663-4189-8c0a-2c17c68e32d8 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:36:00.184024 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:36:00.258674 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c0066-cd47-4aa4-95eb-41bcf7205ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:36:00.260599 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c0066-cd47-4aa4-95eb-41bcf7205ab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:00.262588 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c0066-cd47-4aa4-95eb-41bcf7205ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:00.263227 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c0066-cd47-4aa4-95eb-41bcf7205ab0 admin admin] PATH_INFO: `/v3/projects/9560bf66ad094c3fb3e17783dd59e504` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:00.267018 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c0066-cd47-4aa4-95eb-41bcf7205ab0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9560bf66ad094c3fb3e17783dd59e504)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:36:00.270664 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c0066-cd47-4aa4-95eb-41bcf7205ab0 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:36:00.468801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-c40c0066-cd47-4aa4-95eb-41bcf7205ab0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9560bf66ad094c3fb3e17783dd59e504', 'request_id': 'req-c40c0066-cd47-4aa4-95eb-41bcf7205ab0'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:36:00.493723 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-c40c0066-cd47-4aa4-95eb-41bcf7205ab0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9560bf66ad094c3fb3e17783dd59e504', 'request_id': 'req-c40c0066-cd47-4aa4-95eb-41bcf7205ab0'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:36:00.511571 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-c40c0066-cd47-4aa4-95eb-41bcf7205ab0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9560bf66ad094c3fb3e17783dd59e504', 'request_id': 'req-c40c0066-cd47-4aa4-95eb-41bcf7205ab0'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:36:00.555572 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-c40c0066-cd47-4aa4-95eb-41bcf7205ab0 admin admin] The token cache is being invalidate because project 9560bf66ad094c3fb3e17783dd59e504 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:36:00.555572 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-c40c0066-cd47-4aa4-95eb-41bcf7205ab0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:36:00.556635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-c40c0066-cd47-4aa4-95eb-41bcf7205ab0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:36:00.564784 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1127/2264] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:36:00 2022] DELETE /identity/v3/projects/9560bf66ad094c3fb3e17783dd59e504 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:36:00.566773 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950c785-b583-4b1f-980a-fa390d62e972 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:00.567329 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950c785-b583-4b1f-980a-fa390d62e972 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:00.567615 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950c785-b583-4b1f-980a-fa390d62e972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:00.643612 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-e950c785-b583-4b1f-980a-fa390d62e972 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:36:00.645118 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e950c785-b583-4b1f-980a-fa390d62e972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:36:00.905929 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1138/2265] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:36:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:36:00.908067 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655438f-e917-4226-9699-fccce43f5dc9 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:00.908584 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655438f-e917-4226-9699-fccce43f5dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:00.908876 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655438f-e917-4226-9699-fccce43f5dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:00.978898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-e655438f-e917-4226-9699-fccce43f5dc9 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:36:00.990584 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-e655438f-e917-4226-9699-fccce43f5dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:36:01.024727 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1128/2266] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:36:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:36:01.034505 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950c785-b583-4b1f-980a-fa390d62e972 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:36:01.052097 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950c785-b583-4b1f-980a-fa390d62e972 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:36:01.067899 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01be11f8-7494-4308-bc15-a1f2a1330c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:36:01.070850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be11f8-7494-4308-bc15-a1f2a1330c76 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:01.070850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be11f8-7494-4308-bc15-a1f2a1330c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:01.070850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01be11f8-7494-4308-bc15-a1f2a1330c76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:01.070850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01be11f8-7494-4308-bc15-a1f2a1330c76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:36:01.070850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01be11f8-7494-4308-bc15-a1f2a1330c76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:36:01.074342 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01be11f8-7494-4308-bc15-a1f2a1330c76 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:36:01.139738 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1139/2267] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:36:01.143826 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655438f-e917-4226-9699-fccce43f5dc9 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:36:01.165202 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655438f-e917-4226-9699-fccce43f5dc9 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:36:01.196522 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee699fc-010f-40d0-bd6e-3d01c1c10ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:36:01.196932 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee699fc-010f-40d0-bd6e-3d01c1c10ced admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:01.197214 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee699fc-010f-40d0-bd6e-3d01c1c10ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:01.197500 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee699fc-010f-40d0-bd6e-3d01c1c10ced admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:01.198211 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee699fc-010f-40d0-bd6e-3d01c1c10ced admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:36:01.204110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee699fc-010f-40d0-bd6e-3d01c1c10ced admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:36:01.238838 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20026443-d64b-404b-bb46-a12e9c359ea2 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:01.239132 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20026443-d64b-404b-bb46-a12e9c359ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:01.239415 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20026443-d64b-404b-bb46-a12e9c359ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:01.283187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-20026443-d64b-404b-bb46-a12e9c359ea2 None None] MFA Rules not processed for user `921861792ac246aa8772a636d45489c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:36:01.284716 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-20026443-d64b-404b-bb46-a12e9c359ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:36:01.302720 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1129/2268] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:36:01 2022] POST /identity/v3/projects => generated 373 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:36:01.307480 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee699fc-010f-40d0-bd6e-3d01c1c10ced admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:36:01.367871 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1140/2269] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:36:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:36:01.437592 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee699fc-010f-40d0-bd6e-3d01c1c10ced admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:36:01.447426 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed9650-7250-4ca7-8950-4e8847aa454c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:36:01.448226 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed9650-7250-4ca7-8950-4e8847aa454c admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:01.448437 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed9650-7250-4ca7-8950-4e8847aa454c admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:01.449707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed9650-7250-4ca7-8950-4e8847aa454c admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:01.476200 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed9650-7250-4ca7-8950-4e8847aa454c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:36:01.495919 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed9650-7250-4ca7-8950-4e8847aa454c admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:36:01.628910 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1130/2270] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:36:01 2022] POST /identity/v3/users => generated 287 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:36:01.630179 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20026443-d64b-404b-bb46-a12e9c359ea2 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:36:01.658091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20026443-d64b-404b-bb46-a12e9c359ea2 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:36:01.685223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83273e9-6aa2-4f21-9392-0a86d7030759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:36:01.690624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83273e9-6aa2-4f21-9392-0a86d7030759 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:01.690920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83273e9-6aa2-4f21-9392-0a86d7030759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:01.692231 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83273e9-6aa2-4f21-9392-0a86d7030759 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:01.692231 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83273e9-6aa2-4f21-9392-0a86d7030759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:36:01.693836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83273e9-6aa2-4f21-9392-0a86d7030759 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:36:01.714508 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1141/2271] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:36:01.718091 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed9650-7250-4ca7-8950-4e8847aa454c admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:36:01.730467 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed9650-7250-4ca7-8950-4e8847aa454c admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:36:01.760946 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0892cfcc-6684-469f-91f2-7992603ccad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:36:01.762836 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892cfcc-6684-469f-91f2-7992603ccad3 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:01.764402 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892cfcc-6684-469f-91f2-7992603ccad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:01.764925 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892cfcc-6684-469f-91f2-7992603ccad3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:01.767133 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0892cfcc-6684-469f-91f2-7992603ccad3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:36:01.770885 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0892cfcc-6684-469f-91f2-7992603ccad3 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:36:01.798482 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1131/2272] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:36:01.802439 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83273e9-6aa2-4f21-9392-0a86d7030759 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:36:01.822906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83273e9-6aa2-4f21-9392-0a86d7030759 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:36:01.831258 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d83c4f-1478-4e93-b690-b6378558576b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:36:01.832768 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d83c4f-1478-4e93-b690-b6378558576b admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:01.833183 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d83c4f-1478-4e93-b690-b6378558576b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:01.833469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d83c4f-1478-4e93-b690-b6378558576b admin admin] PATH_INFO: `/v3/projects/18761ad798c7467b9e4e7c862aba52c5/users/c71cce0eb765463da7515cf8ae7b40c7/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:01.877291 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d83c4f-1478-4e93-b690-b6378558576b admin admin] RBAC: Authorizing `identity:create_grant(project_id=18761ad798c7467b9e4e7c862aba52c5, user_id=c71cce0eb765463da7515cf8ae7b40c7, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:36:01.880091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d83c4f-1478-4e93-b690-b6378558576b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:36:01.903821 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1142/2273] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:36:01 2022] PUT /identity/v3/projects/18761ad798c7467b9e4e7c862aba52c5/users/c71cce0eb765463da7515cf8ae7b40c7/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:36:01.916400 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e4bd3-b8be-47dd-9119-16546dacd94a None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:01.916400 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e4bd3-b8be-47dd-9119-16546dacd94a None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:01.916747 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e4bd3-b8be-47dd-9119-16546dacd94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:01.971604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:36:01.981840 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-179e4bd3-b8be-47dd-9119-16546dacd94a None None] MFA Rules not processed for user `c71cce0eb765463da7515cf8ae7b40c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:36:01.983373 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-179e4bd3-b8be-47dd-9119-16546dacd94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:36:02.297926 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1132/2274] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:36:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:36:02.313764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d83c4f-1478-4e93-b690-b6378558576b admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:36:02.315736 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-60d83c4f-1478-4e93-b690-b6378558576b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:36:02.505742 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:36:02.517268 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d83c4f-1478-4e93-b690-b6378558576b admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:36:02.530296 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4f33d-057d-4c43-a27d-bac9ea31b310 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:36:02.531927 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4f33d-057d-4c43-a27d-bac9ea31b310 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:02.533049 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4f33d-057d-4c43-a27d-bac9ea31b310 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:02.533844 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4f33d-057d-4c43-a27d-bac9ea31b310 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:02.550215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d4f33d-057d-4c43-a27d-bac9ea31b310 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:36:02.552012 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d4f33d-057d-4c43-a27d-bac9ea31b310 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:36:02.568240 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1143/2275] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:36:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:36:04.850359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89b4c0-9e20-4caa-93f5-d432f5905217 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:04.851701 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89b4c0-9e20-4caa-93f5-d432f5905217 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:04.852055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c89b4c0-9e20-4caa-93f5-d432f5905217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:04.904047 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-2c89b4c0-9e20-4caa-93f5-d432f5905217 None None] MFA Rules not processed for user `687208d733174bffa2783e6857401f39`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:36:04.905320 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-2c89b4c0-9e20-4caa-93f5-d432f5905217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:36:05.028713 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1133/2276] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:36:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:36:12.561188 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:36:12.828767 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4f33d-057d-4c43-a27d-bac9ea31b310 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:36:12.830707 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-90d4f33d-057d-4c43-a27d-bac9ea31b310 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:36:12.930077 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4f33d-057d-4c43-a27d-bac9ea31b310 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:36:12.935234 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf1b3d-f495-489a-8fa6-798c97464d57 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc0f137e0d24dd2a460689bc17a1b98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:36:12.935887 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf1b3d-f495-489a-8fa6-798c97464d57 service placement] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:12.936192 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf1b3d-f495-489a-8fa6-798c97464d57 service placement] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:12.936480 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf1b3d-f495-489a-8fa6-798c97464d57 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:12.938271 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1dcf1b3d-f495-489a-8fa6-798c97464d57 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:36:12.949766 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcf1b3d-f495-489a-8fa6-798c97464d57 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:36:12.951583 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcf1b3d-f495-489a-8fa6-798c97464d57 service placement] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:36:12.962068 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1144/2277] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:36:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:36:14.684057 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c89b4c0-9e20-4caa-93f5-d432f5905217 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:36:14.755283 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c89b4c0-9e20-4caa-93f5-d432f5905217 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:36:14.761453 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1aa4a4-ac8f-46d8-8d24-14554ffffa63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:36:14.762429 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1aa4a4-ac8f-46d8-8d24-14554ffffa63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:14.762791 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1aa4a4-ac8f-46d8-8d24-14554ffffa63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:14.763212 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1aa4a4-ac8f-46d8-8d24-14554ffffa63 admin admin] PATH_INFO: `/v3/users/c71cce0eb765463da7515cf8ae7b40c7` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:14.764781 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1aa4a4-ac8f-46d8-8d24-14554ffffa63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c71cce0eb765463da7515cf8ae7b40c7)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:36:14.766755 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1aa4a4-ac8f-46d8-8d24-14554ffffa63 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:36:14.838019 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-6c1aa4a4-ac8f-46d8-8d24-14554ffffa63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c71cce0eb765463da7515cf8ae7b40c7'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:36:14.838763 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-6c1aa4a4-ac8f-46d8-8d24-14554ffffa63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c71cce0eb765463da7515cf8ae7b40c7'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:36:14.840967 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:36:14.848229 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-6c1aa4a4-ac8f-46d8-8d24-14554ffffa63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c71cce0eb765463da7515cf8ae7b40c7'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:36:14.868681 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-6c1aa4a4-ac8f-46d8-8d24-14554ffffa63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c71cce0eb765463da7515cf8ae7b40c7'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:36:14.882050 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-6c1aa4a4-ac8f-46d8-8d24-14554ffffa63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c71cce0eb765463da7515cf8ae7b40c7'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:36:14.889719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1134/2278] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:36:14 2022] DELETE /identity/v3/users/c71cce0eb765463da7515cf8ae7b40c7 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:36:14.893969 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820dfa35-5ea9-4cbb-a85f-c53d824679a5 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:14.894477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820dfa35-5ea9-4cbb-a85f-c53d824679a5 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:14.894595 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820dfa35-5ea9-4cbb-a85f-c53d824679a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:14.918107 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-820dfa35-5ea9-4cbb-a85f-c53d824679a5 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:36:14.919571 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-820dfa35-5ea9-4cbb-a85f-c53d824679a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:36:14.982148 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1145/2279] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:36:14.992490 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1aa4a4-ac8f-46d8-8d24-14554ffffa63 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:36:14.994135 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-6c1aa4a4-ac8f-46d8-8d24-14554ffffa63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:36:15.051478 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1aa4a4-ac8f-46d8-8d24-14554ffffa63 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:36:15.056101 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a229f85-3211-438d-8272-77094e456231 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:36:15.056803 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a229f85-3211-438d-8272-77094e456231 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:15.057096 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a229f85-3211-438d-8272-77094e456231 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:15.057380 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a229f85-3211-438d-8272-77094e456231 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:15.066668 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a229f85-3211-438d-8272-77094e456231 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:36:15.068114 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a229f85-3211-438d-8272-77094e456231 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:36:15.078038 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1135/2280] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:36:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:36:15.351915 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820dfa35-5ea9-4cbb-a85f-c53d824679a5 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:36:15.353460 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-820dfa35-5ea9-4cbb-a85f-c53d824679a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:36:15.366178 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820dfa35-5ea9-4cbb-a85f-c53d824679a5 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:36:15.371748 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca106d6-d752-4448-8892-2d97a372c289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:36:15.372703 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca106d6-d752-4448-8892-2d97a372c289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:15.372990 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca106d6-d752-4448-8892-2d97a372c289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:15.373268 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca106d6-d752-4448-8892-2d97a372c289 admin admin] PATH_INFO: `/v3/projects/18761ad798c7467b9e4e7c862aba52c5` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:15.374494 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca106d6-d752-4448-8892-2d97a372c289 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18761ad798c7467b9e4e7c862aba52c5)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:36:15.376610 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca106d6-d752-4448-8892-2d97a372c289 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:36:15.431260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9ca106d6-d752-4448-8892-2d97a372c289 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18761ad798c7467b9e4e7c862aba52c5', 'request_id': 'req-9ca106d6-d752-4448-8892-2d97a372c289'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:36:15.437338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9ca106d6-d752-4448-8892-2d97a372c289 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18761ad798c7467b9e4e7c862aba52c5', 'request_id': 'req-9ca106d6-d752-4448-8892-2d97a372c289'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:36:15.443505 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9ca106d6-d752-4448-8892-2d97a372c289 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18761ad798c7467b9e4e7c862aba52c5', 'request_id': 'req-9ca106d6-d752-4448-8892-2d97a372c289'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:36:15.446692 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:36:15.458760 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9ca106d6-d752-4448-8892-2d97a372c289 admin admin] The token cache is being invalidate because project 18761ad798c7467b9e4e7c862aba52c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:36:15.459052 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9ca106d6-d752-4448-8892-2d97a372c289 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:36:15.459681 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9ca106d6-d752-4448-8892-2d97a372c289 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:36:15.461687 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1146/2281] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:36:15 2022] DELETE /identity/v3/projects/18761ad798c7467b9e4e7c862aba52c5 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:36:16.788560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca07a7-d124-485e-aa38-27d3d84e443d None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:16.789303 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca07a7-d124-485e-aa38-27d3d84e443d None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:16.789303 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca07a7-d124-485e-aa38-27d3d84e443d None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:16.790536 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1136/2282] 10.222.0.16 () {54 vars in 921 bytes} [Mon May 23 12:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:36:16.796727 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d0cd6-69e0-48ff-ba53-ecb2c322eeab None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:16.797184 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d0cd6-69e0-48ff-ba53-ecb2c322eeab None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:16.797317 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d0cd6-69e0-48ff-ba53-ecb2c322eeab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:16.841748 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-c76d0cd6-69e0-48ff-ba53-ecb2c322eeab None None] MFA Rules not processed for user `b83535b443a84275a160cd84cb3ad5a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:36:16.842986 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c76d0cd6-69e0-48ff-ba53-ecb2c322eeab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:36:16.938916 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1147/2283] 10.222.0.16 () {58 vars in 1018 bytes} [Mon May 23 12:36:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:36:16.946947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca07a7-d124-485e-aa38-27d3d84e443d None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:36:16.949114 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-e6ca07a7-d124-485e-aa38-27d3d84e443d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:36:17.058651 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca07a7-d124-485e-aa38-27d3d84e443d None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:36:17.064335 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c55fb-07e6-4f29-936e-d750bce7ccaf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2483a351f5e943ad8c872a98a4717b0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:36:17.065104 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c55fb-07e6-4f29-936e-d750bce7ccaf service glance] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:17.065997 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c55fb-07e6-4f29-936e-d750bce7ccaf service glance] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:17.066361 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c55fb-07e6-4f29-936e-d750bce7ccaf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:17.079344 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c55fb-07e6-4f29-936e-d750bce7ccaf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:36:17.081211 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c55fb-07e6-4f29-936e-d750bce7ccaf service glance] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:36:17.094377 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1137/2284] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:36:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:36:17.179885 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbf7c1-151a-450d-b374-578263d4678b None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:17.180199 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbf7c1-151a-450d-b374-578263d4678b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:17.180504 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbf7c1-151a-450d-b374-578263d4678b None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:17.181956 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1148/2285] 10.222.0.16 () {54 vars in 921 bytes} [Mon May 23 12:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:36:17.187890 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee39f4-6f37-4de1-ac78-86f57a0c3ead None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:17.192534 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee39f4-6f37-4de1-ac78-86f57a0c3ead None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:17.193005 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee39f4-6f37-4de1-ac78-86f57a0c3ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:17.225772 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-6eee39f4-6f37-4de1-ac78-86f57a0c3ead None None] MFA Rules not processed for user `b6677be3ddf647e2b971bb3ba2c34a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:36:17.227040 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-6eee39f4-6f37-4de1-ac78-86f57a0c3ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:36:17.299673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1138/2286] 10.222.0.16 () {58 vars in 1018 bytes} [Mon May 23 12:36:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:36:17.317398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbf7c1-151a-450d-b374-578263d4678b None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:36:17.321211 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-01bbf7c1-151a-450d-b374-578263d4678b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:36:17.336751 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbf7c1-151a-450d-b374-578263d4678b None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:36:17.340918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba1ce2-6670-4cd5-8292-1d4d9d09db69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:36:17.341600 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba1ce2-6670-4cd5-8292-1d4d9d09db69 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:17.341885 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba1ce2-6670-4cd5-8292-1d4d9d09db69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:17.342161 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba1ce2-6670-4cd5-8292-1d4d9d09db69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:17.352601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba1ce2-6670-4cd5-8292-1d4d9d09db69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:36:17.353951 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba1ce2-6670-4cd5-8292-1d4d9d09db69 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:36:17.363342 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1149/2287] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:36:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:36:17.451340 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78b103-08df-4c7f-94c4-0fc20965b1c8 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:17.452090 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78b103-08df-4c7f-94c4-0fc20965b1c8 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:17.452404 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78b103-08df-4c7f-94c4-0fc20965b1c8 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:17.455073 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1139/2288] 10.222.0.16 () {54 vars in 921 bytes} [Mon May 23 12:36:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:36:17.461503 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4a203-4e82-47a1-b9fe-7bab53f309f7 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:17.461828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4a203-4e82-47a1-b9fe-7bab53f309f7 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:17.462321 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4a203-4e82-47a1-b9fe-7bab53f309f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:17.484714 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-1be4a203-4e82-47a1-b9fe-7bab53f309f7 None None] MFA Rules not processed for user `b83535b443a84275a160cd84cb3ad5a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:36:17.485934 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1be4a203-4e82-47a1-b9fe-7bab53f309f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:36:17.495373 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1150/2289] 10.222.0.16 () {58 vars in 1018 bytes} [Mon May 23 12:36:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:36:17.506313 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78b103-08df-4c7f-94c4-0fc20965b1c8 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:36:17.511803 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:36:17.519512 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78b103-08df-4c7f-94c4-0fc20965b1c8 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:36:17.525081 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e08422e-ab59-4441-ae73-ab7d3ff69a6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:36:17.525732 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08422e-ab59-4441-ae73-ab7d3ff69a6c service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:17.526047 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08422e-ab59-4441-ae73-ab7d3ff69a6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:17.526331 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08422e-ab59-4441-ae73-ab7d3ff69a6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:17.536979 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e08422e-ab59-4441-ae73-ab7d3ff69a6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:36:17.539528 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e08422e-ab59-4441-ae73-ab7d3ff69a6c service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:36:17.551703 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1140/2290] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:36:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:36:21.121109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ea7bb-6d22-4eb0-a791-f59f06b77158 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:21.121592 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ea7bb-6d22-4eb0-a791-f59f06b77158 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:21.121682 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ea7bb-6d22-4eb0-a791-f59f06b77158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:21.139931 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-d61ea7bb-6d22-4eb0-a791-f59f06b77158 None None] MFA Rules not processed for user `733f8296ec3d4795a1140c1e1cb6eb95`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:36:21.141185 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d61ea7bb-6d22-4eb0-a791-f59f06b77158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:36:21.198380 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1151/2291] 10.222.0.16 () {58 vars in 1009 bytes} [Mon May 23 12:36:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:36:21.211387 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e08422e-ab59-4441-ae73-ab7d3ff69a6c service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:36:21.213181 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-3e08422e-ab59-4441-ae73-ab7d3ff69a6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:36:21.287091 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e08422e-ab59-4441-ae73-ab7d3ff69a6c service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:36:21.292528 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a27f9d-a767-4f84-a86f-84669021c968 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71fb9a79ac494635804b93d4a0048205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:36:21.293184 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a27f9d-a767-4f84-a86f-84669021c968 service swift] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:21.293467 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a27f9d-a767-4f84-a86f-84669021c968 service swift] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:21.293780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a27f9d-a767-4f84-a86f-84669021c968 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:21.302409 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a27f9d-a767-4f84-a86f-84669021c968 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:36:21.304569 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a27f9d-a767-4f84-a86f-84669021c968 service swift] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:36:21.313068 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1141/2292] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 12:36:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 12:36:31.191244 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:DELETE=1,keystone:SELECT=73 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:36:31.309497 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=37 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:36:46.518566 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4f672-7215-4589-9bae-fe23226b88b5 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:46.519050 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4f672-7215-4589-9bae-fe23226b88b5 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:46.519150 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4f672-7215-4589-9bae-fe23226b88b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:46.537243 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-c7c4f672-7215-4589-9bae-fe23226b88b5 None None] MFA Rules not processed for user `733f8296ec3d4795a1140c1e1cb6eb95`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:36:46.538361 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c7c4f672-7215-4589-9bae-fe23226b88b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:36:46.546161 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1152/2293] 10.222.0.16 () {58 vars in 967 bytes} [Mon May 23 12:36:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:36:46.558983 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a27f9d-a767-4f84-a86f-84669021c968 service swift] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:36:46.569490 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a27f9d-a767-4f84-a86f-84669021c968 service swift] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:36:46.575298 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98d967-c534-4b75-b789-7798018cec71 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71fb9a79ac494635804b93d4a0048205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:36:46.575932 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98d967-c534-4b75-b789-7798018cec71 service swift] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:46.576252 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98d967-c534-4b75-b789-7798018cec71 service swift] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:46.576537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98d967-c534-4b75-b789-7798018cec71 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:46.584904 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c98d967-c534-4b75-b789-7798018cec71 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:36:46.586699 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c98d967-c534-4b75-b789-7798018cec71 service swift] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:36:46.594789 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1142/2294] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 12:36:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 12:36:49.097278 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4f672-7215-4589-9bae-fe23226b88b5 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:36:49.107551 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4f672-7215-4589-9bae-fe23226b88b5 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:36:49.116272 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2de526-ba56-4ec3-9ec2-e9497f21e9f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:36:49.116929 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2de526-ba56-4ec3-9ec2-e9497f21e9f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:36:49.117206 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2de526-ba56-4ec3-9ec2-e9497f21e9f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:36:49.118108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2de526-ba56-4ec3-9ec2-e9497f21e9f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:36:49.120073 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-9b2de526-ba56-4ec3-9ec2-e9497f21e9f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:36:49.182147 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2de526-ba56-4ec3-9ec2-e9497f21e9f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:36:49.183566 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2de526-ba56-4ec3-9ec2-e9497f21e9f8 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:36:49.192951 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1153/2295] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:36:49 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:36:56.591553 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=6 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:36:59.188779 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=33 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:37:08.413501 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf89254-5857-43b4-9cfc-662dc787c166 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:37:08.414067 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf89254-5857-43b4-9cfc-662dc787c166 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:37:08.414164 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf89254-5857-43b4-9cfc-662dc787c166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:37:08.432531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-bbf89254-5857-43b4-9cfc-662dc787c166 None None] MFA Rules not processed for user `ef222fe025534a9e942f8c5c4ed1e3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:37:08.433765 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-bbf89254-5857-43b4-9cfc-662dc787c166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:37:08.492709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1143/2296] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:37:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:37:18.485695 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=25 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:37:43.391025 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b28d6-6352-4b67-9c4b-3a51e089cf3e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:37:43.391580 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b28d6-6352-4b67-9c4b-3a51e089cf3e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:37:43.391580 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b28d6-6352-4b67-9c4b-3a51e089cf3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:37:43.409224 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-dd8b28d6-6352-4b67-9c4b-3a51e089cf3e None None] MFA Rules not processed for user `851454212d14438d80bf8e193fea268e`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:37:43.410502 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-dd8b28d6-6352-4b67-9c4b-3a51e089cf3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:37:43.469507 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1154/2297] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:37:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:37:52.228616 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c76420-c1d1-4d5b-a26c-faca3a367ce0 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:37:52.229230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c76420-c1d1-4d5b-a26c-faca3a367ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:37:52.229230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c76420-c1d1-4d5b-a26c-faca3a367ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:37:52.247610 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-74c76420-c1d1-4d5b-a26c-faca3a367ce0 None None] MFA Rules not processed for user `687208d733174bffa2783e6857401f39`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:37:52.248856 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-74c76420-c1d1-4d5b-a26c-faca3a367ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:37:52.306257 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1144/2298] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:37:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:37:53.462032 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=25 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:38:02.298524 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=25 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:38:05.902714 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c4582-3f51-4769-8c13-5c5896ea5185 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:38:05.903261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c4582-3f51-4769-8c13-5c5896ea5185 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:38:05.903376 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c4582-3f51-4769-8c13-5c5896ea5185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:38:05.921352 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-730c4582-3f51-4769-8c13-5c5896ea5185 None None] MFA Rules not processed for user `771d3e22beff44f7bbc3103fd75a35b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:38:05.922553 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-730c4582-3f51-4769-8c13-5c5896ea5185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:38:05.981445 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1155/2299] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:38:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:38:08.847185 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c76420-c1d1-4d5b-a26c-faca3a367ce0 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:38:08.860063 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c76420-c1d1-4d5b-a26c-faca3a367ce0 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:38:08.865274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a3d58f-1f43-41b1-91c2-e73fb0bd203b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:38:08.866204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3d58f-1f43-41b1-91c2-e73fb0bd203b service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:38:08.866647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3d58f-1f43-41b1-91c2-e73fb0bd203b service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:38:08.867096 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a3d58f-1f43-41b1-91c2-e73fb0bd203b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:38:08.870356 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-24a3d58f-1f43-41b1-91c2-e73fb0bd203b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:38:08.897038 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a3d58f-1f43-41b1-91c2-e73fb0bd203b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:38:08.899790 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a3d58f-1f43-41b1-91c2-e73fb0bd203b service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:38:08.920299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1145/2300] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:38:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:38:09.844503 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c4582-3f51-4769-8c13-5c5896ea5185 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:38:09.846065 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-730c4582-3f51-4769-8c13-5c5896ea5185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:38:09.908207 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c4582-3f51-4769-8c13-5c5896ea5185 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:38:09.914374 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5303c5-9ff0-43e2-aed5-597414ee7380 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:38:09.915409 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5303c5-9ff0-43e2-aed5-597414ee7380 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:38:09.916402 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5303c5-9ff0-43e2-aed5-597414ee7380 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:38:09.916850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5303c5-9ff0-43e2-aed5-597414ee7380 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:38:09.919168 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-9c5303c5-9ff0-43e2-aed5-597414ee7380 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:38:09.933769 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5303c5-9ff0-43e2-aed5-597414ee7380 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:38:09.935219 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5303c5-9ff0-43e2-aed5-597414ee7380 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:38:09.946936 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1156/2301] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:38:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:38:18.908879 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=6 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:38:19.424368 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb186ad-31e9-446b-8b69-e154e34d85e0 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:38:19.424754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb186ad-31e9-446b-8b69-e154e34d85e0 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:38:19.425045 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb186ad-31e9-446b-8b69-e154e34d85e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:38:19.443071 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-dfb186ad-31e9-446b-8b69-e154e34d85e0 None None] MFA Rules not processed for user `ef222fe025534a9e942f8c5c4ed1e3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:38:19.444317 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-dfb186ad-31e9-446b-8b69-e154e34d85e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:38:19.451098 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1146/2302] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:38:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:38:19.757522 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5303c5-9ff0-43e2-aed5-597414ee7380 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:38:19.767916 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5303c5-9ff0-43e2-aed5-597414ee7380 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:38:19.771713 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0fad6-c384-4ac9-a8ad-fafafa1fa0ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:38:19.772453 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0fad6-c384-4ac9-a8ad-fafafa1fa0ac service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:38:19.772780 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0fad6-c384-4ac9-a8ad-fafafa1fa0ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:38:19.773114 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0fad6-c384-4ac9-a8ad-fafafa1fa0ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:38:19.775692 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d1b0fad6-c384-4ac9-a8ad-fafafa1fa0ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:38:19.785570 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b0fad6-c384-4ac9-a8ad-fafafa1fa0ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:38:19.786992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b0fad6-c384-4ac9-a8ad-fafafa1fa0ac service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:38:19.795223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1157/2303] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:38:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:38:20.111899 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb186ad-31e9-446b-8b69-e154e34d85e0 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:38:20.113871 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-dfb186ad-31e9-446b-8b69-e154e34d85e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:38:20.186755 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb186ad-31e9-446b-8b69-e154e34d85e0 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:38:20.191769 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19f3a4-8acf-4201-b600-0fb57675405b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc0f137e0d24dd2a460689bc17a1b98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:38:20.192453 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19f3a4-8acf-4201-b600-0fb57675405b service placement] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:38:20.192740 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19f3a4-8acf-4201-b600-0fb57675405b service placement] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:38:20.193020 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19f3a4-8acf-4201-b600-0fb57675405b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:38:20.194782 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-7e19f3a4-8acf-4201-b600-0fb57675405b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:38:20.207118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19f3a4-8acf-4201-b600-0fb57675405b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:38:20.208751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19f3a4-8acf-4201-b600-0fb57675405b service placement] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:38:20.218549 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1147/2304] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:38:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:38:22.704475 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0fad6-c384-4ac9-a8ad-fafafa1fa0ac service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:38:22.706257 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d1b0fad6-c384-4ac9-a8ad-fafafa1fa0ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:38:22.720573 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0fad6-c384-4ac9-a8ad-fafafa1fa0ac service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:38:22.724334 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e70b0e2-8b6c-46bd-be0d-329708babe92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:38:22.724988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70b0e2-8b6c-46bd-be0d-329708babe92 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:38:22.725463 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70b0e2-8b6c-46bd-be0d-329708babe92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:38:22.725764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e70b0e2-8b6c-46bd-be0d-329708babe92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:38:22.728535 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-0e70b0e2-8b6c-46bd-be0d-329708babe92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:38:22.740131 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e70b0e2-8b6c-46bd-be0d-329708babe92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:38:22.741644 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e70b0e2-8b6c-46bd-be0d-329708babe92 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:38:22.752085 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1158/2305] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:38:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:38:30.214441 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=35 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:38:30.492632 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19f3a4-8acf-4201-b600-0fb57675405b service placement] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:38:30.494354 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-7e19f3a4-8acf-4201-b600-0fb57675405b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:38:30.509787 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19f3a4-8acf-4201-b600-0fb57675405b service placement] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:38:30.515768 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c8656c-e520-40ba-a850-bb46e884a8ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:38:30.516550 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8656c-e520-40ba-a850-bb46e884a8ff service nova] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:38:30.516836 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8656c-e520-40ba-a850-bb46e884a8ff service nova] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:38:30.517112 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c8656c-e520-40ba-a850-bb46e884a8ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:38:30.518871 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-48c8656c-e520-40ba-a850-bb46e884a8ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:38:30.531307 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c8656c-e520-40ba-a850-bb46e884a8ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:38:30.532936 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c8656c-e520-40ba-a850-bb46e884a8ff service nova] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:38:30.543791 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1148/2306] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:38:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:38:32.747395 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=63 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:38:38.720157 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e70b0e2-8b6c-46bd-be0d-329708babe92 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:38:38.733415 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e70b0e2-8b6c-46bd-be0d-329708babe92 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:38:38.738850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa50c9d-2760-4cb1-b233-124ba5c56950 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:38:38.739509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa50c9d-2760-4cb1-b233-124ba5c56950 service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:38:38.739790 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa50c9d-2760-4cb1-b233-124ba5c56950 service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:38:38.740065 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa50c9d-2760-4cb1-b233-124ba5c56950 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:38:38.741842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-eaa50c9d-2760-4cb1-b233-124ba5c56950 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:38:38.753226 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa50c9d-2760-4cb1-b233-124ba5c56950 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:38:38.754807 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa50c9d-2760-4cb1-b233-124ba5c56950 service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:38:38.764622 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1159/2307] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:38:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:38:39.622126 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c8656c-e520-40ba-a850-bb46e884a8ff service nova] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:38:39.623723 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-48c8656c-e520-40ba-a850-bb46e884a8ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:38:39.639244 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c8656c-e520-40ba-a850-bb46e884a8ff service nova] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:38:39.644702 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ad19b-bd5d-4073-aa18-0af1f5fd2215 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:38:39.645372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ad19b-bd5d-4073-aa18-0af1f5fd2215 service nova] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:38:39.645659 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ad19b-bd5d-4073-aa18-0af1f5fd2215 service nova] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:38:39.645938 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ad19b-bd5d-4073-aa18-0af1f5fd2215 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:38:39.647657 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-752ad19b-bd5d-4073-aa18-0af1f5fd2215 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:38:39.658667 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752ad19b-bd5d-4073-aa18-0af1f5fd2215 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:38:39.660295 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752ad19b-bd5d-4073-aa18-0af1f5fd2215 service nova] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:38:39.669918 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1149/2308] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:38:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:38:48.759681 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=6 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:38:49.607786 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa50c9d-2760-4cb1-b233-124ba5c56950 service nova] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:38:49.626964 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa50c9d-2760-4cb1-b233-124ba5c56950 service nova] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:38:49.636414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b000d-1090-4713-89b9-c11f7d6884db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:38:49.637358 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b000d-1090-4713-89b9-c11f7d6884db service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:38:49.638498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b000d-1090-4713-89b9-c11f7d6884db service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:38:49.639022 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b000d-1090-4713-89b9-c11f7d6884db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:38:49.641359 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-787b000d-1090-4713-89b9-c11f7d6884db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:38:49.655190 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787b000d-1090-4713-89b9-c11f7d6884db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:38:49.656852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787b000d-1090-4713-89b9-c11f7d6884db service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:38:49.665299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=12 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:38:49.672518 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1160/2309] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:38:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:38:56.058206 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ad19b-bd5d-4073-aa18-0af1f5fd2215 service nova] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:38:56.067379 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ad19b-bd5d-4073-aa18-0af1f5fd2215 service nova] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:38:56.070999 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb598a7-592e-4ceb-a21f-3324fe200c0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:38:56.071633 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb598a7-592e-4ceb-a21f-3324fe200c0d service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:38:56.071922 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb598a7-592e-4ceb-a21f-3324fe200c0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:38:56.072276 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb598a7-592e-4ceb-a21f-3324fe200c0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:38:56.074016 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-5cb598a7-592e-4ceb-a21f-3324fe200c0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:38:56.136838 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb598a7-592e-4ceb-a21f-3324fe200c0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:38:56.138298 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb598a7-592e-4ceb-a21f-3324fe200c0d service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:38:56.147987 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1150/2310] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:38:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:38:59.666642 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=6 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:00.099675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b000d-1090-4713-89b9-c11f7d6884db service nova] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:00.101477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-787b000d-1090-4713-89b9-c11f7d6884db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:00.116107 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b000d-1090-4713-89b9-c11f7d6884db service nova] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:00.120717 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69c31f-10a9-4712-b894-e49d12f5f31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:00.121591 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69c31f-10a9-4712-b894-e49d12f5f31e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:00.121901 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69c31f-10a9-4712-b894-e49d12f5f31e admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:00.122183 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69c31f-10a9-4712-b894-e49d12f5f31e admin admin] PATH_INFO: `/v3/users/687208d733174bffa2783e6857401f39` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:00.124838 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb69c31f-10a9-4712-b894-e49d12f5f31e admin admin] RBAC: Authorizing `identity:delete_user(user_id=687208d733174bffa2783e6857401f39)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:00.126576 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb69c31f-10a9-4712-b894-e49d12f5f31e admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:00.173284 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cb69c31f-10a9-4712-b894-e49d12f5f31e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '687208d733174bffa2783e6857401f39'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:00.184799 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cb69c31f-10a9-4712-b894-e49d12f5f31e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '687208d733174bffa2783e6857401f39'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:00.192661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cb69c31f-10a9-4712-b894-e49d12f5f31e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '687208d733174bffa2783e6857401f39'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:00.193282 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cb69c31f-10a9-4712-b894-e49d12f5f31e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '687208d733174bffa2783e6857401f39'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:00.193892 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cb69c31f-10a9-4712-b894-e49d12f5f31e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '687208d733174bffa2783e6857401f39'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:00.207418 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1161/2311] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:39:00 2022] DELETE /identity/v3/users/687208d733174bffa2783e6857401f39 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:00.380427 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb598a7-592e-4ceb-a21f-3324fe200c0d service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:00.391520 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb598a7-592e-4ceb-a21f-3324fe200c0d service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:00.453659 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623ab7a-34d6-4937-8b3f-8e01844b18e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:00.454814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623ab7a-34d6-4937-8b3f-8e01844b18e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:00.455140 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623ab7a-34d6-4937-8b3f-8e01844b18e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:00.455453 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623ab7a-34d6-4937-8b3f-8e01844b18e9 admin admin] PATH_INFO: `/v3/projects/dcc45b8bab17498cadda8d64b3a76780` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:00.456762 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1623ab7a-34d6-4937-8b3f-8e01844b18e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcc45b8bab17498cadda8d64b3a76780)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:00.458776 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1623ab7a-34d6-4937-8b3f-8e01844b18e9 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:00.512921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-1623ab7a-34d6-4937-8b3f-8e01844b18e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcc45b8bab17498cadda8d64b3a76780', 'request_id': 'req-1623ab7a-34d6-4937-8b3f-8e01844b18e9'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:00.523267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-1623ab7a-34d6-4937-8b3f-8e01844b18e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcc45b8bab17498cadda8d64b3a76780', 'request_id': 'req-1623ab7a-34d6-4937-8b3f-8e01844b18e9'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:00.533018 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-1623ab7a-34d6-4937-8b3f-8e01844b18e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcc45b8bab17498cadda8d64b3a76780', 'request_id': 'req-1623ab7a-34d6-4937-8b3f-8e01844b18e9'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:00.551081 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-1623ab7a-34d6-4937-8b3f-8e01844b18e9 admin admin] The token cache is being invalidate because project dcc45b8bab17498cadda8d64b3a76780 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:39:00.551425 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-1623ab7a-34d6-4937-8b3f-8e01844b18e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:00.552202 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-1623ab7a-34d6-4937-8b3f-8e01844b18e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:00.554967 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1151/2312] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:39:00 2022] DELETE /identity/v3/projects/dcc45b8bab17498cadda8d64b3a76780 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:00.566152 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71fca2-d054-4f2f-bfce-8118fb70d939 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:00.566785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71fca2-d054-4f2f-bfce-8118fb70d939 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:00.567086 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71fca2-d054-4f2f-bfce-8118fb70d939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:00.588899 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-ea71fca2-d054-4f2f-bfce-8118fb70d939 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:00.590491 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ea71fca2-d054-4f2f-bfce-8118fb70d939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:00.662516 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1162/2313] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:39:00.670668 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe22dc-1f75-4dd9-b6b3-09a692aaaae2 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:00.671011 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe22dc-1f75-4dd9-b6b3-09a692aaaae2 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:00.671293 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe22dc-1f75-4dd9-b6b3-09a692aaaae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:00.692498 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-aebe22dc-1f75-4dd9-b6b3-09a692aaaae2 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:00.694016 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-aebe22dc-1f75-4dd9-b6b3-09a692aaaae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:00.707561 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1152/2314] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:39:00.709212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71fca2-d054-4f2f-bfce-8118fb70d939 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:00.719879 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71fca2-d054-4f2f-bfce-8118fb70d939 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:00.724476 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61871502-3f78-46ef-97f3-5be7275735c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:00.725171 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61871502-3f78-46ef-97f3-5be7275735c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:00.725472 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61871502-3f78-46ef-97f3-5be7275735c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:00.725750 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61871502-3f78-46ef-97f3-5be7275735c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:00.726511 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61871502-3f78-46ef-97f3-5be7275735c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:39:00.726828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61871502-3f78-46ef-97f3-5be7275735c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:00.728523 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61871502-3f78-46ef-97f3-5be7275735c8 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:00.742615 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1163/2315] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:39:00.747131 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe22dc-1f75-4dd9-b6b3-09a692aaaae2 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:00.757402 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe22dc-1f75-4dd9-b6b3-09a692aaaae2 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:00.762682 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd2a2f-bbd2-44a7-b074-84f81939660e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:00.763552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd2a2f-bbd2-44a7-b074-84f81939660e admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:00.764078 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd2a2f-bbd2-44a7-b074-84f81939660e admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:00.764560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd2a2f-bbd2-44a7-b074-84f81939660e admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:00.765329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd2a2f-bbd2-44a7-b074-84f81939660e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:00.767313 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd2a2f-bbd2-44a7-b074-84f81939660e admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:00.781796 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:00.797095 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1153/2316] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:39:00 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:39:00.800538 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61871502-3f78-46ef-97f3-5be7275735c8 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:00.810367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61871502-3f78-46ef-97f3-5be7275735c8 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:00.866724 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0866c-c00a-4c8d-b001-eb1caba549b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:00.867390 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0866c-c00a-4c8d-b001-eb1caba549b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:00.867672 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0866c-c00a-4c8d-b001-eb1caba549b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:00.867970 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0866c-c00a-4c8d-b001-eb1caba549b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:00.868686 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0866c-c00a-4c8d-b001-eb1caba549b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:00.870603 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0866c-c00a-4c8d-b001-eb1caba549b2 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:00.877368 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:00.892586 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1164/2317] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:39:00 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:39:00.896664 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd2a2f-bbd2-44a7-b074-84f81939660e admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:00.907363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd2a2f-bbd2-44a7-b074-84f81939660e admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:00.912098 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89135352-3b50-4fc3-a38a-5c50808df037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:00.912787 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89135352-3b50-4fc3-a38a-5c50808df037 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:00.913073 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89135352-3b50-4fc3-a38a-5c50808df037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:00.913354 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89135352-3b50-4fc3-a38a-5c50808df037 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:00.914080 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89135352-3b50-4fc3-a38a-5c50808df037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:00.915685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89135352-3b50-4fc3-a38a-5c50808df037 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:00.926745 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1154/2318] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:39:00.930886 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0866c-c00a-4c8d-b001-eb1caba549b2 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:00.941661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0866c-c00a-4c8d-b001-eb1caba549b2 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:00.947018 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a878e09b-12a8-4977-bb26-68f6b34725c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:00.947591 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878e09b-12a8-4977-bb26-68f6b34725c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:00.947880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878e09b-12a8-4977-bb26-68f6b34725c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:00.948150 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878e09b-12a8-4977-bb26-68f6b34725c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:00.948949 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a878e09b-12a8-4977-bb26-68f6b34725c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:00.950552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a878e09b-12a8-4977-bb26-68f6b34725c6 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:00.961164 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1165/2319] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:39:00.965948 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89135352-3b50-4fc3-a38a-5c50808df037 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:00.977240 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89135352-3b50-4fc3-a38a-5c50808df037 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:00.981893 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ef23e-e4f5-4cc7-80bf-ce7377c60864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:00.983364 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ef23e-e4f5-4cc7-80bf-ce7377c60864 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:00.983704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ef23e-e4f5-4cc7-80bf-ce7377c60864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:00.983984 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ef23e-e4f5-4cc7-80bf-ce7377c60864 admin admin] PATH_INFO: `/v3/projects/35ead38007d34d59aeddc017beb98a61/users/f57930f78bfc41ada38a1d3d4c7d5370/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:01.000605 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ef23e-e4f5-4cc7-80bf-ce7377c60864 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35ead38007d34d59aeddc017beb98a61, user_id=f57930f78bfc41ada38a1d3d4c7d5370, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:01.003366 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ef23e-e4f5-4cc7-80bf-ce7377c60864 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:01.017059 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1155/2320] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:39:00 2022] PUT /identity/v3/projects/35ead38007d34d59aeddc017beb98a61/users/f57930f78bfc41ada38a1d3d4c7d5370/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:01.027590 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192832d2-8252-42c1-8abd-ab29db9cf2ab None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:01.027934 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192832d2-8252-42c1-8abd-ab29db9cf2ab None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:01.028265 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192832d2-8252-42c1-8abd-ab29db9cf2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:01.057727 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-192832d2-8252-42c1-8abd-ab29db9cf2ab None None] MFA Rules not processed for user `f57930f78bfc41ada38a1d3d4c7d5370`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:01.058941 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-192832d2-8252-42c1-8abd-ab29db9cf2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:01.126686 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1166/2321] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:39:01.131573 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ef23e-e4f5-4cc7-80bf-ce7377c60864 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:01.198204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ef23e-e4f5-4cc7-80bf-ce7377c60864 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:01.203513 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08220e59-29c6-4127-8f24-97adbe57835f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:01.204197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08220e59-29c6-4127-8f24-97adbe57835f admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:01.204494 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08220e59-29c6-4127-8f24-97adbe57835f admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:01.204779 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08220e59-29c6-4127-8f24-97adbe57835f admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:01.205488 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08220e59-29c6-4127-8f24-97adbe57835f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:01.207419 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08220e59-29c6-4127-8f24-97adbe57835f admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:01.232014 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1156/2322] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:39:01 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:39:01.237167 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192832d2-8252-42c1-8abd-ab29db9cf2ab None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:01.248960 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08220e59-29c6-4127-8f24-97adbe57835f admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:01.250220 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192832d2-8252-42c1-8abd-ab29db9cf2ab None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:01.250836 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-08220e59-29c6-4127-8f24-97adbe57835f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:01.308743 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb84d4e-7d03-4863-96c3-9e996679ca28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:01.309422 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb84d4e-7d03-4863-96c3-9e996679ca28 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:01.309701 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb84d4e-7d03-4863-96c3-9e996679ca28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:01.309981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb84d4e-7d03-4863-96c3-9e996679ca28 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:01.311461 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb84d4e-7d03-4863-96c3-9e996679ca28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:01.313634 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb84d4e-7d03-4863-96c3-9e996679ca28 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:01.329124 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08220e59-29c6-4127-8f24-97adbe57835f admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:01.334750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f2d22-5158-40ac-9130-a99eb842736d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:01.338020 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1167/2323] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:39:01 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:39:01.338173 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f2d22-5158-40ac-9130-a99eb842736d service nova] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:01.338173 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f2d22-5158-40ac-9130-a99eb842736d service nova] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:01.338173 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f2d22-5158-40ac-9130-a99eb842736d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:01.338173 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-808f2d22-5158-40ac-9130-a99eb842736d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:01.340797 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb84d4e-7d03-4863-96c3-9e996679ca28 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:01.350656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb84d4e-7d03-4863-96c3-9e996679ca28 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:01.355725 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0f7685-8c9f-4814-b03e-a82147f86e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:01.356420 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f7685-8c9f-4814-b03e-a82147f86e34 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:01.356714 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f7685-8c9f-4814-b03e-a82147f86e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:01.356995 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f7685-8c9f-4814-b03e-a82147f86e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:01.358354 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0f7685-8c9f-4814-b03e-a82147f86e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:01.359893 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0f7685-8c9f-4814-b03e-a82147f86e34 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:01.370332 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1168/2324] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:39:01.375213 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0f7685-8c9f-4814-b03e-a82147f86e34 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:01.383721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:01.385895 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0f7685-8c9f-4814-b03e-a82147f86e34 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:01.391910 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6b614-4996-4590-8180-1a5a4d31acc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:01.393057 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6b614-4996-4590-8180-1a5a4d31acc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:01.393348 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6b614-4996-4590-8180-1a5a4d31acc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:01.393631 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6b614-4996-4590-8180-1a5a4d31acc2 admin admin] PATH_INFO: `/v3/projects/c270f95a528e4fe880642f4c664685cb/users/c7f4b99d29ec44adb37beb5770e31fe4/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:01.401066 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f2d22-5158-40ac-9130-a99eb842736d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:01.402717 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f2d22-5158-40ac-9130-a99eb842736d service nova] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:01.409212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6b614-4996-4590-8180-1a5a4d31acc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c270f95a528e4fe880642f4c664685cb, user_id=c7f4b99d29ec44adb37beb5770e31fe4, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:01.411867 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6b614-4996-4590-8180-1a5a4d31acc2 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:01.412439 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1157/2325] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:39:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:39:01.427360 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1169/2326] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:39:01 2022] PUT /identity/v3/projects/c270f95a528e4fe880642f4c664685cb/users/c7f4b99d29ec44adb37beb5770e31fe4/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:39:01.431456 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f2d22-5158-40ac-9130-a99eb842736d service nova] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:01.442377 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f2d22-5158-40ac-9130-a99eb842736d service nova] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:01.507828 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2cf5c-c395-4204-b047-793f61ba2c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:01.508546 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2cf5c-c395-4204-b047-793f61ba2c20 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:01.508835 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2cf5c-c395-4204-b047-793f61ba2c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:01.509118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2cf5c-c395-4204-b047-793f61ba2c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:01.509853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2cf5c-c395-4204-b047-793f61ba2c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:01.511458 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2cf5c-c395-4204-b047-793f61ba2c20 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:01.525336 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1158/2327] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:39:01.526577 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6b614-4996-4590-8180-1a5a4d31acc2 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:01.537835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6b614-4996-4590-8180-1a5a4d31acc2 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:01.542517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a8a32-64ea-47e8-81cf-78436d0d49c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:01.543440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a8a32-64ea-47e8-81cf-78436d0d49c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:01.543727 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a8a32-64ea-47e8-81cf-78436d0d49c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:01.544019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a8a32-64ea-47e8-81cf-78436d0d49c0 admin admin] PATH_INFO: `/v3/projects/c270f95a528e4fe880642f4c664685cb/users/c7f4b99d29ec44adb37beb5770e31fe4/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:01.546205 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233a8a32-64ea-47e8-81cf-78436d0d49c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c270f95a528e4fe880642f4c664685cb, user_id=c7f4b99d29ec44adb37beb5770e31fe4, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:01.549050 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233a8a32-64ea-47e8-81cf-78436d0d49c0 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:01.560998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1170/2328] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:39:01 2022] PUT /identity/v3/projects/c270f95a528e4fe880642f4c664685cb/users/c7f4b99d29ec44adb37beb5770e31fe4/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:39:01.572876 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795262c-5da9-4def-8060-5ca8452f8fca None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:01.573177 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795262c-5da9-4def-8060-5ca8452f8fca None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:01.573452 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795262c-5da9-4def-8060-5ca8452f8fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:01.604395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-b795262c-5da9-4def-8060-5ca8452f8fca None None] MFA Rules not processed for user `c7f4b99d29ec44adb37beb5770e31fe4`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:01.605766 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-b795262c-5da9-4def-8060-5ca8452f8fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:01.704815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1159/2329] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:01 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:39:01.716506 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a8a32-64ea-47e8-81cf-78436d0d49c0 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:01.718957 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-233a8a32-64ea-47e8-81cf-78436d0d49c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:01.731874 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:01.796751 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a8a32-64ea-47e8-81cf-78436d0d49c0 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:01.800578 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c19026-87d0-420a-b316-b1acf61c3870 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:01.801229 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c19026-87d0-420a-b316-b1acf61c3870 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:01.801512 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c19026-87d0-420a-b316-b1acf61c3870 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:01.801790 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c19026-87d0-420a-b316-b1acf61c3870 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:01.862912 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c19026-87d0-420a-b316-b1acf61c3870 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:01.864408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c19026-87d0-420a-b316-b1acf61c3870 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:01.873519 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1171/2330] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:39:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:39:07.502823 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795262c-5da9-4def-8060-5ca8452f8fca None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:07.512354 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795262c-5da9-4def-8060-5ca8452f8fca None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:07.516708 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44cf52-e387-450f-82f9-3f1f6d10057a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:07.517342 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44cf52-e387-450f-82f9-3f1f6d10057a service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:07.517630 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44cf52-e387-450f-82f9-3f1f6d10057a service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:07.517914 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44cf52-e387-450f-82f9-3f1f6d10057a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:07.527027 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b44cf52-e387-450f-82f9-3f1f6d10057a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:07.528426 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b44cf52-e387-450f-82f9-3f1f6d10057a service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:07.538984 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1160/2331] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:39:07 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:39:11.869115 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=44 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:17.534456 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=77 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:22.374877 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c19026-87d0-420a-b316-b1acf61c3870 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:22.376829 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b7c19026-87d0-420a-b316-b1acf61c3870 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:22.488309 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c19026-87d0-420a-b316-b1acf61c3870 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:22.493940 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ae6101-61d3-43cc-8b9e-7625d51e052a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:22.494588 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae6101-61d3-43cc-8b9e-7625d51e052a service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:22.494869 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae6101-61d3-43cc-8b9e-7625d51e052a service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:22.495149 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ae6101-61d3-43cc-8b9e-7625d51e052a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:22.496926 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-32ae6101-61d3-43cc-8b9e-7625d51e052a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:22.510152 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ae6101-61d3-43cc-8b9e-7625d51e052a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:22.511721 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ae6101-61d3-43cc-8b9e-7625d51e052a service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:22.522950 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1172/2332] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:39:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:39:32.431369 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44cf52-e387-450f-82f9-3f1f6d10057a service cinder] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:32.498110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44cf52-e387-450f-82f9-3f1f6d10057a service cinder] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:32.502763 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f419ae-473e-44a6-b887-e90adfa7115f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:32.503686 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f419ae-473e-44a6-b887-e90adfa7115f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:32.503977 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f419ae-473e-44a6-b887-e90adfa7115f admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:32.504285 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f419ae-473e-44a6-b887-e90adfa7115f admin admin] PATH_INFO: `/v3/users/f57930f78bfc41ada38a1d3d4c7d5370` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:32.506171 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f419ae-473e-44a6-b887-e90adfa7115f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f57930f78bfc41ada38a1d3d4c7d5370)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:32.508134 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f419ae-473e-44a6-b887-e90adfa7115f admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:32.517664 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=34 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:32.558699 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-86f419ae-473e-44a6-b887-e90adfa7115f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f57930f78bfc41ada38a1d3d4c7d5370'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:32.559372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-86f419ae-473e-44a6-b887-e90adfa7115f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f57930f78bfc41ada38a1d3d4c7d5370'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:32.566836 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-86f419ae-473e-44a6-b887-e90adfa7115f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f57930f78bfc41ada38a1d3d4c7d5370'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:32.580295 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-86f419ae-473e-44a6-b887-e90adfa7115f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f57930f78bfc41ada38a1d3d4c7d5370'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:32.588595 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-86f419ae-473e-44a6-b887-e90adfa7115f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f57930f78bfc41ada38a1d3d4c7d5370'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:32.594732 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1161/2333] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:39:32 2022] DELETE /identity/v3/users/f57930f78bfc41ada38a1d3d4c7d5370 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:32.599445 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b29b8e-2967-4b10-8dd8-b11fddb4182e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:32.599750 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b29b8e-2967-4b10-8dd8-b11fddb4182e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:32.600038 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b29b8e-2967-4b10-8dd8-b11fddb4182e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:32.617687 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-37b29b8e-2967-4b10-8dd8-b11fddb4182e None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:32.618909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-37b29b8e-2967-4b10-8dd8-b11fddb4182e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:32.677447 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1173/2334] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:39:32.687115 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f419ae-473e-44a6-b887-e90adfa7115f admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:32.688575 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-86f419ae-473e-44a6-b887-e90adfa7115f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:32.746299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f419ae-473e-44a6-b887-e90adfa7115f admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:32.750033 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49b450-41f9-4900-b19b-64bb5a7f58e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:32.750674 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49b450-41f9-4900-b19b-64bb5a7f58e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:32.750962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49b450-41f9-4900-b19b-64bb5a7f58e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:32.751238 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49b450-41f9-4900-b19b-64bb5a7f58e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:32.760389 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49b450-41f9-4900-b19b-64bb5a7f58e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:32.761854 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49b450-41f9-4900-b19b-64bb5a7f58e7 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:32.773563 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1162/2335] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:39:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:39:33.121586 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b29b8e-2967-4b10-8dd8-b11fddb4182e None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:33.123302 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-37b29b8e-2967-4b10-8dd8-b11fddb4182e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:33.136028 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b29b8e-2967-4b10-8dd8-b11fddb4182e None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:33.140836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df9ab17-edf4-4778-9372-a7323bf6c787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:33.141780 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9ab17-edf4-4778-9372-a7323bf6c787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:33.142136 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9ab17-edf4-4778-9372-a7323bf6c787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:33.142507 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9ab17-edf4-4778-9372-a7323bf6c787 admin admin] PATH_INFO: `/v3/projects/35ead38007d34d59aeddc017beb98a61` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:33.143777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df9ab17-edf4-4778-9372-a7323bf6c787 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35ead38007d34d59aeddc017beb98a61)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:33.146633 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df9ab17-edf4-4778-9372-a7323bf6c787 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:33.197555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-0df9ab17-edf4-4778-9372-a7323bf6c787 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35ead38007d34d59aeddc017beb98a61', 'request_id': 'req-0df9ab17-edf4-4778-9372-a7323bf6c787'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:33.203523 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-0df9ab17-edf4-4778-9372-a7323bf6c787 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35ead38007d34d59aeddc017beb98a61', 'request_id': 'req-0df9ab17-edf4-4778-9372-a7323bf6c787'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:33.217014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-0df9ab17-edf4-4778-9372-a7323bf6c787 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35ead38007d34d59aeddc017beb98a61', 'request_id': 'req-0df9ab17-edf4-4778-9372-a7323bf6c787'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:33.230253 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-0df9ab17-edf4-4778-9372-a7323bf6c787 admin admin] The token cache is being invalidate because project 35ead38007d34d59aeddc017beb98a61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:39:33.230692 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-0df9ab17-edf4-4778-9372-a7323bf6c787 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:33.231337 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-0df9ab17-edf4-4778-9372-a7323bf6c787 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:33.232939 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1174/2336] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:39:33 2022] DELETE /identity/v3/projects/35ead38007d34d59aeddc017beb98a61 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:33.237705 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49b450-41f9-4900-b19b-64bb5a7f58e7 service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:33.250459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49b450-41f9-4900-b19b-64bb5a7f58e7 service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:33.274866 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:33.316501 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef8192-e7ee-4fbd-a81c-edfedef6b2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:33.317468 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef8192-e7ee-4fbd-a81c-edfedef6b2f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:33.317840 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef8192-e7ee-4fbd-a81c-edfedef6b2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:33.318127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef8192-e7ee-4fbd-a81c-edfedef6b2f1 admin admin] PATH_INFO: `/v3/users/c7f4b99d29ec44adb37beb5770e31fe4` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:33.319307 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef8192-e7ee-4fbd-a81c-edfedef6b2f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7f4b99d29ec44adb37beb5770e31fe4)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:33.321279 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef8192-e7ee-4fbd-a81c-edfedef6b2f1 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:33.368134 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-88ef8192-e7ee-4fbd-a81c-edfedef6b2f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7f4b99d29ec44adb37beb5770e31fe4'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:33.368822 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-88ef8192-e7ee-4fbd-a81c-edfedef6b2f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7f4b99d29ec44adb37beb5770e31fe4'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:33.375616 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-88ef8192-e7ee-4fbd-a81c-edfedef6b2f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7f4b99d29ec44adb37beb5770e31fe4'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:33.387969 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-88ef8192-e7ee-4fbd-a81c-edfedef6b2f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7f4b99d29ec44adb37beb5770e31fe4'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:33.394612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-88ef8192-e7ee-4fbd-a81c-edfedef6b2f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7f4b99d29ec44adb37beb5770e31fe4'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:33.402318 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1163/2337] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:39:33 2022] DELETE /identity/v3/users/c7f4b99d29ec44adb37beb5770e31fe4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:39:33.718699 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df9ab17-edf4-4778-9372-a7323bf6c787 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:33.720275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-0df9ab17-edf4-4778-9372-a7323bf6c787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:33.738381 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df9ab17-edf4-4778-9372-a7323bf6c787 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:33.817797 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650154bd-6cdd-4ab7-9bf5-47ef66d9d52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:33.818672 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650154bd-6cdd-4ab7-9bf5-47ef66d9d52b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:33.818962 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650154bd-6cdd-4ab7-9bf5-47ef66d9d52b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:33.819249 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650154bd-6cdd-4ab7-9bf5-47ef66d9d52b admin admin] PATH_INFO: `/v3/projects/c270f95a528e4fe880642f4c664685cb` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:33.820466 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650154bd-6cdd-4ab7-9bf5-47ef66d9d52b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c270f95a528e4fe880642f4c664685cb)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:33.822362 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650154bd-6cdd-4ab7-9bf5-47ef66d9d52b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:33.854737 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:33.872569 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-650154bd-6cdd-4ab7-9bf5-47ef66d9d52b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c270f95a528e4fe880642f4c664685cb', 'request_id': 'req-650154bd-6cdd-4ab7-9bf5-47ef66d9d52b'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:33.877900 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-650154bd-6cdd-4ab7-9bf5-47ef66d9d52b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c270f95a528e4fe880642f4c664685cb', 'request_id': 'req-650154bd-6cdd-4ab7-9bf5-47ef66d9d52b'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:33.884088 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-650154bd-6cdd-4ab7-9bf5-47ef66d9d52b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c270f95a528e4fe880642f4c664685cb', 'request_id': 'req-650154bd-6cdd-4ab7-9bf5-47ef66d9d52b'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:33.897428 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-650154bd-6cdd-4ab7-9bf5-47ef66d9d52b admin admin] The token cache is being invalidate because project c270f95a528e4fe880642f4c664685cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:39:33.897713 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-650154bd-6cdd-4ab7-9bf5-47ef66d9d52b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:33.898365 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-650154bd-6cdd-4ab7-9bf5-47ef66d9d52b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:33.900015 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1175/2338] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:39:33 2022] DELETE /identity/v3/projects/c270f95a528e4fe880642f4c664685cb => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:33.905154 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ec12a-5872-4ea7-84fa-216c3d317a31 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:33.905614 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ec12a-5872-4ea7-84fa-216c3d317a31 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:33.905726 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ec12a-5872-4ea7-84fa-216c3d317a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:33.925801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-6d5ec12a-5872-4ea7-84fa-216c3d317a31 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:33.927064 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-6d5ec12a-5872-4ea7-84fa-216c3d317a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:33.996397 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1164/2339] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:39:34.004659 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef49a67-0208-48b3-9684-aced1afb78bc None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:34.004954 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef49a67-0208-48b3-9684-aced1afb78bc None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:34.005235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef49a67-0208-48b3-9684-aced1afb78bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:34.024818 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-fef49a67-0208-48b3-9684-aced1afb78bc None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:34.026033 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-fef49a67-0208-48b3-9684-aced1afb78bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:34.035079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1176/2340] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:39:34.040158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ec12a-5872-4ea7-84fa-216c3d317a31 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:34.050799 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ec12a-5872-4ea7-84fa-216c3d317a31 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:34.057326 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab84280-c63b-40ae-9b0e-6d0aafa4f8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:34.058090 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab84280-c63b-40ae-9b0e-6d0aafa4f8e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:34.060871 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab84280-c63b-40ae-9b0e-6d0aafa4f8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:34.061173 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab84280-c63b-40ae-9b0e-6d0aafa4f8e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:34.061928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab84280-c63b-40ae-9b0e-6d0aafa4f8e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:39:34.062446 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab84280-c63b-40ae-9b0e-6d0aafa4f8e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:34.063860 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab84280-c63b-40ae-9b0e-6d0aafa4f8e9 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:34.080514 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1165/2341] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:39:34.081641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef49a67-0208-48b3-9684-aced1afb78bc None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:34.092279 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef49a67-0208-48b3-9684-aced1afb78bc None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:34.101295 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d22c5-1f9a-4d22-9de9-2e207cc2613b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:34.102026 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d22c5-1f9a-4d22-9de9-2e207cc2613b admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:34.102315 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d22c5-1f9a-4d22-9de9-2e207cc2613b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:34.102649 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d22c5-1f9a-4d22-9de9-2e207cc2613b admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:34.104817 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d22c5-1f9a-4d22-9de9-2e207cc2613b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:34.106752 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d22c5-1f9a-4d22-9de9-2e207cc2613b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:34.162978 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1177/2342] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:39:34 2022] POST /identity/v3/projects => generated 393 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:39:34.167307 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab84280-c63b-40ae-9b0e-6d0aafa4f8e9 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:34.180347 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab84280-c63b-40ae-9b0e-6d0aafa4f8e9 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:34.214438 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:34.312627 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd506f9c-da59-4d35-81d7-73e1da2678b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:34.313467 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd506f9c-da59-4d35-81d7-73e1da2678b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:34.313743 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd506f9c-da59-4d35-81d7-73e1da2678b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:34.314098 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd506f9c-da59-4d35-81d7-73e1da2678b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:34.314870 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd506f9c-da59-4d35-81d7-73e1da2678b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:34.316953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd506f9c-da59-4d35-81d7-73e1da2678b8 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:34.344979 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1166/2343] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:39:34 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:39:34.351899 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d22c5-1f9a-4d22-9de9-2e207cc2613b admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:34.362566 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d22c5-1f9a-4d22-9de9-2e207cc2613b admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:34.367946 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c0f90-d3fe-4ba0-880b-cdb10570ec5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:34.368649 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c0f90-d3fe-4ba0-880b-cdb10570ec5e admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:34.368934 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c0f90-d3fe-4ba0-880b-cdb10570ec5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:34.369216 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c0f90-d3fe-4ba0-880b-cdb10570ec5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:34.369944 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c0f90-d3fe-4ba0-880b-cdb10570ec5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:34.371534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c0f90-d3fe-4ba0-880b-cdb10570ec5e admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:34.384358 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1178/2344] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:39:34.385665 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd506f9c-da59-4d35-81d7-73e1da2678b8 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:34.396794 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd506f9c-da59-4d35-81d7-73e1da2678b8 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:34.401473 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3649cb-fcda-4939-a2d7-8b3bfefdf0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:34.402228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3649cb-fcda-4939-a2d7-8b3bfefdf0c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:34.402519 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3649cb-fcda-4939-a2d7-8b3bfefdf0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:34.402802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3649cb-fcda-4939-a2d7-8b3bfefdf0c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:34.403619 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3649cb-fcda-4939-a2d7-8b3bfefdf0c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:34.405457 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3649cb-fcda-4939-a2d7-8b3bfefdf0c4 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:34.416734 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1167/2345] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:39:34.420859 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c0f90-d3fe-4ba0-880b-cdb10570ec5e admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:34.432473 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c0f90-d3fe-4ba0-880b-cdb10570ec5e admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:34.437261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c0d5f-ee3d-4848-b759-4f8bcf53d7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:34.438154 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c0d5f-ee3d-4848-b759-4f8bcf53d7dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:34.438445 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c0d5f-ee3d-4848-b759-4f8bcf53d7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:34.438725 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c0d5f-ee3d-4848-b759-4f8bcf53d7dc admin admin] PATH_INFO: `/v3/projects/3f6aeeabb8a643dba4c288ee7858d2c9/users/6558370cdd28473b8855f314d6bd7f62/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:34.458124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c0d5f-ee3d-4848-b759-4f8bcf53d7dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f6aeeabb8a643dba4c288ee7858d2c9, user_id=6558370cdd28473b8855f314d6bd7f62, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:34.461218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c0d5f-ee3d-4848-b759-4f8bcf53d7dc admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:34.478437 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1179/2346] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:39:34 2022] PUT /identity/v3/projects/3f6aeeabb8a643dba4c288ee7858d2c9/users/6558370cdd28473b8855f314d6bd7f62/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:34.492050 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7facc-c631-40f6-857e-71aee5d2de3b None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:34.492620 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7facc-c631-40f6-857e-71aee5d2de3b None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:34.493065 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7facc-c631-40f6-857e-71aee5d2de3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:34.535003 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-63c7facc-c631-40f6-857e-71aee5d2de3b None None] MFA Rules not processed for user `6558370cdd28473b8855f314d6bd7f62`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:34.541171 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-63c7facc-c631-40f6-857e-71aee5d2de3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:34.621305 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1168/2347] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:39:34.625801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c0d5f-ee3d-4848-b759-4f8bcf53d7dc admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:34.689458 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c0d5f-ee3d-4848-b759-4f8bcf53d7dc admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:34.694175 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26525dd5-e3c7-4ff9-859e-127ebceec8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:34.694835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26525dd5-e3c7-4ff9-859e-127ebceec8c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:34.695114 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26525dd5-e3c7-4ff9-859e-127ebceec8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:34.695389 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26525dd5-e3c7-4ff9-859e-127ebceec8c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:34.696064 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26525dd5-e3c7-4ff9-859e-127ebceec8c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:34.697993 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26525dd5-e3c7-4ff9-859e-127ebceec8c7 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:34.723282 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1180/2348] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:39:34 2022] POST /identity/v3/projects => generated 391 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:39:34.727966 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7facc-c631-40f6-857e-71aee5d2de3b None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:34.738525 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7facc-c631-40f6-857e-71aee5d2de3b None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:34.794252 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24b262-dfc9-495d-95f7-e7692c1890bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:34.794831 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24b262-dfc9-495d-95f7-e7692c1890bc admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:34.795115 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24b262-dfc9-495d-95f7-e7692c1890bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:34.795394 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24b262-dfc9-495d-95f7-e7692c1890bc admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:34.796087 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24b262-dfc9-495d-95f7-e7692c1890bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:34.798035 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24b262-dfc9-495d-95f7-e7692c1890bc admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:34.823458 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1169/2349] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:39:34 2022] POST /identity/v3/users => generated 302 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:39:34.825511 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26525dd5-e3c7-4ff9-859e-127ebceec8c7 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:34.841480 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26525dd5-e3c7-4ff9-859e-127ebceec8c7 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:34.849220 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a480d012-6291-4fa5-8a71-2491e982e60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:34.850266 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480d012-6291-4fa5-8a71-2491e982e60e admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:34.850710 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480d012-6291-4fa5-8a71-2491e982e60e admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:34.851139 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a480d012-6291-4fa5-8a71-2491e982e60e admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:34.852316 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a480d012-6291-4fa5-8a71-2491e982e60e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:34.854914 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a480d012-6291-4fa5-8a71-2491e982e60e admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:34.870159 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1181/2350] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:39:34.875123 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24b262-dfc9-495d-95f7-e7692c1890bc admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:34.887705 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24b262-dfc9-495d-95f7-e7692c1890bc admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:34.894786 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb604f63-5d88-4af4-a89a-f100e629378d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:34.898409 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb604f63-5d88-4af4-a89a-f100e629378d admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:34.899254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb604f63-5d88-4af4-a89a-f100e629378d admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:34.899933 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb604f63-5d88-4af4-a89a-f100e629378d admin admin] PATH_INFO: `/v3/projects/f2d446e2c55443c595d6fdf8559a3639/users/8dfbcdf08671492199e9306cca2a46c0/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:34.918630 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb604f63-5d88-4af4-a89a-f100e629378d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2d446e2c55443c595d6fdf8559a3639, user_id=8dfbcdf08671492199e9306cca2a46c0, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:34.921464 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb604f63-5d88-4af4-a89a-f100e629378d admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:34.937516 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1170/2351] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:39:34 2022] PUT /identity/v3/projects/f2d446e2c55443c595d6fdf8559a3639/users/8dfbcdf08671492199e9306cca2a46c0/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:39:34.941484 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a480d012-6291-4fa5-8a71-2491e982e60e admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:34.952149 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a480d012-6291-4fa5-8a71-2491e982e60e admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:34.964599 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:35.021810 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ed225-0105-4f34-bf91-9dcc21c368d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:35.022607 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ed225-0105-4f34-bf91-9dcc21c368d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:35.022900 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ed225-0105-4f34-bf91-9dcc21c368d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:35.023189 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ed225-0105-4f34-bf91-9dcc21c368d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:35.023923 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9ed225-0105-4f34-bf91-9dcc21c368d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:35.025584 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9ed225-0105-4f34-bf91-9dcc21c368d4 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:35.035712 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1182/2352] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:39:35.040306 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb604f63-5d88-4af4-a89a-f100e629378d admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:35.046230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:35.051848 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb604f63-5d88-4af4-a89a-f100e629378d admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:35.057331 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b10a36-09e3-4e72-86ea-58be6bc7df01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:35.058269 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b10a36-09e3-4e72-86ea-58be6bc7df01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:35.058604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b10a36-09e3-4e72-86ea-58be6bc7df01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:35.058882 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b10a36-09e3-4e72-86ea-58be6bc7df01 admin admin] PATH_INFO: `/v3/projects/f2d446e2c55443c595d6fdf8559a3639/users/8dfbcdf08671492199e9306cca2a46c0/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:35.060924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b10a36-09e3-4e72-86ea-58be6bc7df01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2d446e2c55443c595d6fdf8559a3639, user_id=8dfbcdf08671492199e9306cca2a46c0, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:35.063661 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b10a36-09e3-4e72-86ea-58be6bc7df01 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:35.074729 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1171/2353] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:39:35 2022] PUT /identity/v3/projects/f2d446e2c55443c595d6fdf8559a3639/users/8dfbcdf08671492199e9306cca2a46c0/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:35.086822 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c06043-1b61-4e2e-b44c-2c590122e36e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:35.087114 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c06043-1b61-4e2e-b44c-2c590122e36e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:35.087396 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c06043-1b61-4e2e-b44c-2c590122e36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:35.113578 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-79c06043-1b61-4e2e-b44c-2c590122e36e None None] MFA Rules not processed for user `8dfbcdf08671492199e9306cca2a46c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:35.115080 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-79c06043-1b61-4e2e-b44c-2c590122e36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:35.184389 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1183/2354] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:35 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:39:35.195949 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b10a36-09e3-4e72-86ea-58be6bc7df01 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:35.197790 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-11b10a36-09e3-4e72-86ea-58be6bc7df01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:35.258460 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b10a36-09e3-4e72-86ea-58be6bc7df01 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:35.262006 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd8120-e03f-4a89-95d2-3b60f1530f64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:35.262654 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd8120-e03f-4a89-95d2-3b60f1530f64 service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:35.262938 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd8120-e03f-4a89-95d2-3b60f1530f64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:35.263258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd8120-e03f-4a89-95d2-3b60f1530f64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:35.272553 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cd8120-e03f-4a89-95d2-3b60f1530f64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:35.274034 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cd8120-e03f-4a89-95d2-3b60f1530f64 service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:35.283296 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1172/2355] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:39:35 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:39:35.329913 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c06043-1b61-4e2e-b44c-2c590122e36e None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:35.339501 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c06043-1b61-4e2e-b44c-2c590122e36e None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:35.343313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd610b-7d1b-4d80-89ce-946fadde3891 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:35.343962 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd610b-7d1b-4d80-89ce-946fadde3891 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:35.344289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd610b-7d1b-4d80-89ce-946fadde3891 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:35.344555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd610b-7d1b-4d80-89ce-946fadde3891 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:35.420477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dd610b-7d1b-4d80-89ce-946fadde3891 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:35.423123 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dd610b-7d1b-4d80-89ce-946fadde3891 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:35.435689 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1184/2356] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:39:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:39:38.907415 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd8120-e03f-4a89-95d2-3b60f1530f64 service cinder] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:38.972014 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd8120-e03f-4a89-95d2-3b60f1530f64 service cinder] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:38.977439 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee4e03-b206-49c1-9aad-1347f076345a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:38.978383 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee4e03-b206-49c1-9aad-1347f076345a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:38.978666 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee4e03-b206-49c1-9aad-1347f076345a admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:38.978947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee4e03-b206-49c1-9aad-1347f076345a admin admin] PATH_INFO: `/v3/users/6558370cdd28473b8855f314d6bd7f62` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:38.980197 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ee4e03-b206-49c1-9aad-1347f076345a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6558370cdd28473b8855f314d6bd7f62)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:38.982136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ee4e03-b206-49c1-9aad-1347f076345a admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:39.042668 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-45ee4e03-b206-49c1-9aad-1347f076345a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6558370cdd28473b8855f314d6bd7f62'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:39.043329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-45ee4e03-b206-49c1-9aad-1347f076345a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6558370cdd28473b8855f314d6bd7f62'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:39.050358 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-45ee4e03-b206-49c1-9aad-1347f076345a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6558370cdd28473b8855f314d6bd7f62'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:39.063386 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-45ee4e03-b206-49c1-9aad-1347f076345a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6558370cdd28473b8855f314d6bd7f62'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:39.072563 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-45ee4e03-b206-49c1-9aad-1347f076345a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6558370cdd28473b8855f314d6bd7f62'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:39.079565 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1173/2357] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:39:38 2022] DELETE /identity/v3/users/6558370cdd28473b8855f314d6bd7f62 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:39.084678 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce6206-e891-4607-bf42-a426848c0026 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:39.085204 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce6206-e891-4607-bf42-a426848c0026 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:39.085302 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce6206-e891-4607-bf42-a426848c0026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:39.103604 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-43ce6206-e891-4607-bf42-a426848c0026 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:39.105106 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-43ce6206-e891-4607-bf42-a426848c0026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:39.135425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:39.165743 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1185/2358] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:39:39.175239 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee4e03-b206-49c1-9aad-1347f076345a admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:39.176843 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-45ee4e03-b206-49c1-9aad-1347f076345a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:39.190672 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:39.238542 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee4e03-b206-49c1-9aad-1347f076345a admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:39.242404 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a66a2-9551-46b1-8c9f-02252aff6f0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:39.243054 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a66a2-9551-46b1-8c9f-02252aff6f0a service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:39.243343 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a66a2-9551-46b1-8c9f-02252aff6f0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:39.243627 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a66a2-9551-46b1-8c9f-02252aff6f0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:39.253560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a66a2-9551-46b1-8c9f-02252aff6f0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:39.255355 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a66a2-9551-46b1-8c9f-02252aff6f0a service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:39.265563 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1174/2359] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:39:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:39:39.822430 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce6206-e891-4607-bf42-a426848c0026 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:39.825193 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-43ce6206-e891-4607-bf42-a426848c0026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:39.838602 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce6206-e891-4607-bf42-a426848c0026 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:39.844093 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d60d167-2724-42b7-a46d-724e3fccf423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:39.845250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60d167-2724-42b7-a46d-724e3fccf423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:39.845583 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60d167-2724-42b7-a46d-724e3fccf423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:39.846116 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60d167-2724-42b7-a46d-724e3fccf423 admin admin] PATH_INFO: `/v3/projects/3f6aeeabb8a643dba4c288ee7858d2c9` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:39.847391 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d60d167-2724-42b7-a46d-724e3fccf423 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f6aeeabb8a643dba4c288ee7858d2c9)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:39.851563 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d60d167-2724-42b7-a46d-724e3fccf423 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:39.903719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-7d60d167-2724-42b7-a46d-724e3fccf423 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f6aeeabb8a643dba4c288ee7858d2c9', 'request_id': 'req-7d60d167-2724-42b7-a46d-724e3fccf423'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:39.909502 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-7d60d167-2724-42b7-a46d-724e3fccf423 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f6aeeabb8a643dba4c288ee7858d2c9', 'request_id': 'req-7d60d167-2724-42b7-a46d-724e3fccf423'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:39.916040 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-7d60d167-2724-42b7-a46d-724e3fccf423 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f6aeeabb8a643dba4c288ee7858d2c9', 'request_id': 'req-7d60d167-2724-42b7-a46d-724e3fccf423'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:39.929472 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-7d60d167-2724-42b7-a46d-724e3fccf423 admin admin] The token cache is being invalidate because project 3f6aeeabb8a643dba4c288ee7858d2c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:39:39.929895 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-7d60d167-2724-42b7-a46d-724e3fccf423 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:39.930564 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-7d60d167-2724-42b7-a46d-724e3fccf423 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:39.932147 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1186/2360] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:39:39 2022] DELETE /identity/v3/projects/3f6aeeabb8a643dba4c288ee7858d2c9 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:39.936110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a66a2-9551-46b1-8c9f-02252aff6f0a service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:39.946815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a66a2-9551-46b1-8c9f-02252aff6f0a service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:40.007511 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d7990-24aa-4744-bfe2-0efaeaea4ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:40.008419 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d7990-24aa-4744-bfe2-0efaeaea4ed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:40.008706 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d7990-24aa-4744-bfe2-0efaeaea4ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:40.008988 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d7990-24aa-4744-bfe2-0efaeaea4ed3 admin admin] PATH_INFO: `/v3/users/8dfbcdf08671492199e9306cca2a46c0` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:40.010163 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d7990-24aa-4744-bfe2-0efaeaea4ed3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dfbcdf08671492199e9306cca2a46c0)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:40.012093 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d7990-24aa-4744-bfe2-0efaeaea4ed3 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:40.058674 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-db8d7990-24aa-4744-bfe2-0efaeaea4ed3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dfbcdf08671492199e9306cca2a46c0'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:40.059532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-db8d7990-24aa-4744-bfe2-0efaeaea4ed3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dfbcdf08671492199e9306cca2a46c0'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:40.066486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-db8d7990-24aa-4744-bfe2-0efaeaea4ed3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dfbcdf08671492199e9306cca2a46c0'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:40.079324 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-db8d7990-24aa-4744-bfe2-0efaeaea4ed3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dfbcdf08671492199e9306cca2a46c0'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:40.087617 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-db8d7990-24aa-4744-bfe2-0efaeaea4ed3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dfbcdf08671492199e9306cca2a46c0'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:40.093766 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1175/2361] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:39:39 2022] DELETE /identity/v3/users/8dfbcdf08671492199e9306cca2a46c0 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:40.641322 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d60d167-2724-42b7-a46d-724e3fccf423 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:40.643376 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-7d60d167-2724-42b7-a46d-724e3fccf423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:40.657421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d60d167-2724-42b7-a46d-724e3fccf423 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:40.750042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20184d-54ac-4114-a668-9f3fffa5def7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:40.751155 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20184d-54ac-4114-a668-9f3fffa5def7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:40.751591 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20184d-54ac-4114-a668-9f3fffa5def7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:40.752036 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20184d-54ac-4114-a668-9f3fffa5def7 admin admin] PATH_INFO: `/v3/projects/f2d446e2c55443c595d6fdf8559a3639` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:40.753431 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20184d-54ac-4114-a668-9f3fffa5def7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2d446e2c55443c595d6fdf8559a3639)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:40.755985 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20184d-54ac-4114-a668-9f3fffa5def7 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:40.809239 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ab20184d-54ac-4114-a668-9f3fffa5def7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2d446e2c55443c595d6fdf8559a3639', 'request_id': 'req-ab20184d-54ac-4114-a668-9f3fffa5def7'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:40.815865 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ab20184d-54ac-4114-a668-9f3fffa5def7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2d446e2c55443c595d6fdf8559a3639', 'request_id': 'req-ab20184d-54ac-4114-a668-9f3fffa5def7'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:40.818769 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:40.824645 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ab20184d-54ac-4114-a668-9f3fffa5def7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2d446e2c55443c595d6fdf8559a3639', 'request_id': 'req-ab20184d-54ac-4114-a668-9f3fffa5def7'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:40.842639 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ab20184d-54ac-4114-a668-9f3fffa5def7 admin admin] The token cache is being invalidate because project f2d446e2c55443c595d6fdf8559a3639 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:39:40.843684 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ab20184d-54ac-4114-a668-9f3fffa5def7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:40.843684 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ab20184d-54ac-4114-a668-9f3fffa5def7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:40.845391 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1187/2362] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:39:40 2022] DELETE /identity/v3/projects/f2d446e2c55443c595d6fdf8559a3639 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:40.851850 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269ce45-07c9-4730-a287-a14b6dac32ed None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:40.852243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269ce45-07c9-4730-a287-a14b6dac32ed None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:40.852645 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269ce45-07c9-4730-a287-a14b6dac32ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:40.875832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-c269ce45-07c9-4730-a287-a14b6dac32ed None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:40.877215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-c269ce45-07c9-4730-a287-a14b6dac32ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:40.921652 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:40.976428 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1176/2363] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:39:40.986054 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02cc54-8049-4f21-bf29-08fdf7abfb91 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:40.986418 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02cc54-8049-4f21-bf29-08fdf7abfb91 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:40.986636 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02cc54-8049-4f21-bf29-08fdf7abfb91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:41.015715 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-ee02cc54-8049-4f21-bf29-08fdf7abfb91 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:41.017432 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ee02cc54-8049-4f21-bf29-08fdf7abfb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:41.032016 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c269ce45-07c9-4730-a287-a14b6dac32ed None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:41.039253 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1188/2364] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:39:41.045187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c269ce45-07c9-4730-a287-a14b6dac32ed None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:41.052936 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de250d5a-db86-43be-8c4f-34efc9e6e23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:41.053630 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de250d5a-db86-43be-8c4f-34efc9e6e23f admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:41.053920 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de250d5a-db86-43be-8c4f-34efc9e6e23f admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:41.054211 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de250d5a-db86-43be-8c4f-34efc9e6e23f admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:41.055139 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de250d5a-db86-43be-8c4f-34efc9e6e23f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:39:41.055257 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de250d5a-db86-43be-8c4f-34efc9e6e23f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:41.057394 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de250d5a-db86-43be-8c4f-34efc9e6e23f admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:41.072480 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1177/2365] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:39:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:39:41.075500 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02cc54-8049-4f21-bf29-08fdf7abfb91 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:41.086904 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02cc54-8049-4f21-bf29-08fdf7abfb91 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:41.091795 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899439a5-9365-49b0-bb5f-94ba8dca69a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:41.092594 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899439a5-9365-49b0-bb5f-94ba8dca69a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:41.092953 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899439a5-9365-49b0-bb5f-94ba8dca69a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:41.093307 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899439a5-9365-49b0-bb5f-94ba8dca69a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:41.094121 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899439a5-9365-49b0-bb5f-94ba8dca69a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:41.096139 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899439a5-9365-49b0-bb5f-94ba8dca69a9 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:41.128485 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1189/2366] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:39:41 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:39:41.133449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de250d5a-db86-43be-8c4f-34efc9e6e23f admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:41.144090 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de250d5a-db86-43be-8c4f-34efc9e6e23f admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:41.229016 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e942e-9145-48a5-b2be-73daa0ec21fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:41.231915 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e942e-9145-48a5-b2be-73daa0ec21fe admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:41.232698 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e942e-9145-48a5-b2be-73daa0ec21fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:41.233338 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e942e-9145-48a5-b2be-73daa0ec21fe admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:41.237141 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6e942e-9145-48a5-b2be-73daa0ec21fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:41.240126 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6e942e-9145-48a5-b2be-73daa0ec21fe admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:41.294381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1178/2367] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:39:41 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:39:41.298225 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899439a5-9365-49b0-bb5f-94ba8dca69a9 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:41.319389 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899439a5-9365-49b0-bb5f-94ba8dca69a9 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:41.326576 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdefb03-6727-43ce-98e6-5c8a8d1430a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:41.327614 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdefb03-6727-43ce-98e6-5c8a8d1430a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:41.327911 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdefb03-6727-43ce-98e6-5c8a8d1430a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:41.328217 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdefb03-6727-43ce-98e6-5c8a8d1430a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:41.329720 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdefb03-6727-43ce-98e6-5c8a8d1430a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:41.332072 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdefb03-6727-43ce-98e6-5c8a8d1430a4 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:41.352747 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1190/2368] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:39:41.363077 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e942e-9145-48a5-b2be-73daa0ec21fe admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:41.392560 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e942e-9145-48a5-b2be-73daa0ec21fe admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:41.410413 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3583a4e-573e-4201-8cde-54482867f01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:41.411978 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3583a4e-573e-4201-8cde-54482867f01d admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:41.412404 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3583a4e-573e-4201-8cde-54482867f01d admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:41.412793 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3583a4e-573e-4201-8cde-54482867f01d admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:41.418045 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3583a4e-573e-4201-8cde-54482867f01d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:41.421646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3583a4e-573e-4201-8cde-54482867f01d admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:41.461438 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1179/2369] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:39:41.466233 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdefb03-6727-43ce-98e6-5c8a8d1430a4 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:41.485603 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdefb03-6727-43ce-98e6-5c8a8d1430a4 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:41.500663 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4deabeb-a392-4354-8b7a-34589d6a8d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:41.501587 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4deabeb-a392-4354-8b7a-34589d6a8d16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:41.501872 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4deabeb-a392-4354-8b7a-34589d6a8d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:41.502154 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4deabeb-a392-4354-8b7a-34589d6a8d16 admin admin] PATH_INFO: `/v3/projects/2078e6fcf0eb4f76989c4a7c908a0bc7/users/9b5780a469fb4a38b2f8cd07d259dba6/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:41.542365 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4deabeb-a392-4354-8b7a-34589d6a8d16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2078e6fcf0eb4f76989c4a7c908a0bc7, user_id=9b5780a469fb4a38b2f8cd07d259dba6, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:41.545223 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4deabeb-a392-4354-8b7a-34589d6a8d16 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:41.578091 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1191/2370] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:39:41 2022] PUT /identity/v3/projects/2078e6fcf0eb4f76989c4a7c908a0bc7/users/9b5780a469fb4a38b2f8cd07d259dba6/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:39:41.580415 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d321c0a-0a84-4770-9e7d-3e954cbd2efe None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:41.581152 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d321c0a-0a84-4770-9e7d-3e954cbd2efe None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:41.581643 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d321c0a-0a84-4770-9e7d-3e954cbd2efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:41.623147 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-1d321c0a-0a84-4770-9e7d-3e954cbd2efe None None] MFA Rules not processed for user `9b5780a469fb4a38b2f8cd07d259dba6`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:41.624833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-1d321c0a-0a84-4770-9e7d-3e954cbd2efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:41.720291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1180/2371] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:39:41.731826 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4deabeb-a392-4354-8b7a-34589d6a8d16 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:41.734133 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e4deabeb-a392-4354-8b7a-34589d6a8d16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:41.820896 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4deabeb-a392-4354-8b7a-34589d6a8d16 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:41.825572 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc8122-9bae-4d20-9f02-56c81d709366 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:41.826358 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc8122-9bae-4d20-9f02-56c81d709366 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:41.826743 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc8122-9bae-4d20-9f02-56c81d709366 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:41.827127 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc8122-9bae-4d20-9f02-56c81d709366 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:41.838337 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc8122-9bae-4d20-9f02-56c81d709366 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:41.839981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc8122-9bae-4d20-9f02-56c81d709366 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:41.849718 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1192/2372] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:39:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:39:41.868976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d321c0a-0a84-4770-9e7d-3e954cbd2efe None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:41.909810 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:41.954030 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d321c0a-0a84-4770-9e7d-3e954cbd2efe None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:41.959060 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cded8bd-7776-46ab-a823-b9f366efc980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:41.959970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cded8bd-7776-46ab-a823-b9f366efc980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:41.960288 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cded8bd-7776-46ab-a823-b9f366efc980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:41.960573 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cded8bd-7776-46ab-a823-b9f366efc980 admin admin] PATH_INFO: `/v3/users/9b5780a469fb4a38b2f8cd07d259dba6` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:41.962716 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cded8bd-7776-46ab-a823-b9f366efc980 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b5780a469fb4a38b2f8cd07d259dba6)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:41.964895 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cded8bd-7776-46ab-a823-b9f366efc980 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:42.067591 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-5cded8bd-7776-46ab-a823-b9f366efc980 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b5780a469fb4a38b2f8cd07d259dba6'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:42.070134 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-5cded8bd-7776-46ab-a823-b9f366efc980 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b5780a469fb4a38b2f8cd07d259dba6'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:42.080075 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc8122-9bae-4d20-9f02-56c81d709366 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:42.081661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-84bc8122-9bae-4d20-9f02-56c81d709366 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:42.089522 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-5cded8bd-7776-46ab-a823-b9f366efc980 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b5780a469fb4a38b2f8cd07d259dba6'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:42.113440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-5cded8bd-7776-46ab-a823-b9f366efc980 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b5780a469fb4a38b2f8cd07d259dba6'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:42.126874 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-5cded8bd-7776-46ab-a823-b9f366efc980 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b5780a469fb4a38b2f8cd07d259dba6'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:42.141258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1181/2373] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:39:41 2022] DELETE /identity/v3/users/9b5780a469fb4a38b2f8cd07d259dba6 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:42.146447 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c735f-5698-4ed0-8314-b54e66c4bf99 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:42.146824 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c735f-5698-4ed0-8314-b54e66c4bf99 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:42.147116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307c735f-5698-4ed0-8314-b54e66c4bf99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:42.178196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-307c735f-5698-4ed0-8314-b54e66c4bf99 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:42.182420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-307c735f-5698-4ed0-8314-b54e66c4bf99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:42.227126 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc8122-9bae-4d20-9f02-56c81d709366 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:42.265524 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:42.295656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1182/2374] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:39:42.302569 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c735f-5698-4ed0-8314-b54e66c4bf99 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:42.305483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-307c735f-5698-4ed0-8314-b54e66c4bf99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:42.319123 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ccb8ac-f087-4622-b666-5fece658cf27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:42.319836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ccb8ac-f087-4622-b666-5fece658cf27 service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:42.320220 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ccb8ac-f087-4622-b666-5fece658cf27 service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:42.320729 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ccb8ac-f087-4622-b666-5fece658cf27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:42.323111 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-62ccb8ac-f087-4622-b666-5fece658cf27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:42.339384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ccb8ac-f087-4622-b666-5fece658cf27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:42.341426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ccb8ac-f087-4622-b666-5fece658cf27 service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:42.357009 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1193/2375] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:39:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:39:42.368518 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:42.374019 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307c735f-5698-4ed0-8314-b54e66c4bf99 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:42.380427 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a8f44-08d1-4c93-bbfa-f40489375278 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:42.381733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a8f44-08d1-4c93-bbfa-f40489375278 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:42.382579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a8f44-08d1-4c93-bbfa-f40489375278 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:42.383194 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a8f44-08d1-4c93-bbfa-f40489375278 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:42.398581 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603a8f44-08d1-4c93-bbfa-f40489375278 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:42.400725 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603a8f44-08d1-4c93-bbfa-f40489375278 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:42.424678 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1183/2376] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:39:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:39:42.950512 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ccb8ac-f087-4622-b666-5fece658cf27 service nova] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:42.952098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-62ccb8ac-f087-4622-b666-5fece658cf27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:42.985558 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ccb8ac-f087-4622-b666-5fece658cf27 service nova] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:42.997217 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef41f91-8601-4351-a79a-0dc748cd6368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:42.998376 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef41f91-8601-4351-a79a-0dc748cd6368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:42.998519 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef41f91-8601-4351-a79a-0dc748cd6368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:42.998773 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef41f91-8601-4351-a79a-0dc748cd6368 admin admin] PATH_INFO: `/v3/projects/2078e6fcf0eb4f76989c4a7c908a0bc7` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:43.001652 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef41f91-8601-4351-a79a-0dc748cd6368 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2078e6fcf0eb4f76989c4a7c908a0bc7)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:43.014000 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef41f91-8601-4351-a79a-0dc748cd6368 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:43.113077 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-0ef41f91-8601-4351-a79a-0dc748cd6368 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2078e6fcf0eb4f76989c4a7c908a0bc7', 'request_id': 'req-0ef41f91-8601-4351-a79a-0dc748cd6368'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:43.121560 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-0ef41f91-8601-4351-a79a-0dc748cd6368 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2078e6fcf0eb4f76989c4a7c908a0bc7', 'request_id': 'req-0ef41f91-8601-4351-a79a-0dc748cd6368'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:43.129432 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-0ef41f91-8601-4351-a79a-0dc748cd6368 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2078e6fcf0eb4f76989c4a7c908a0bc7', 'request_id': 'req-0ef41f91-8601-4351-a79a-0dc748cd6368'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:43.158115 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-0ef41f91-8601-4351-a79a-0dc748cd6368 admin admin] The token cache is being invalidate because project 2078e6fcf0eb4f76989c4a7c908a0bc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:39:43.159662 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-0ef41f91-8601-4351-a79a-0dc748cd6368 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:43.161363 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-0ef41f91-8601-4351-a79a-0dc748cd6368 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:43.164868 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1194/2377] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:39:42 2022] DELETE /identity/v3/projects/2078e6fcf0eb4f76989c4a7c908a0bc7 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:43.170030 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43622ce-04e0-49e1-b337-df50fc2d08b2 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:43.170659 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43622ce-04e0-49e1-b337-df50fc2d08b2 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:43.170808 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43622ce-04e0-49e1-b337-df50fc2d08b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:43.197735 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-b43622ce-04e0-49e1-b337-df50fc2d08b2 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:43.198990 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-b43622ce-04e0-49e1-b337-df50fc2d08b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:43.301586 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1184/2378] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:39:43.309925 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30145d20-cc51-40bf-93d6-3dbffe56ff4f None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:43.310217 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30145d20-cc51-40bf-93d6-3dbffe56ff4f None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:43.310488 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30145d20-cc51-40bf-93d6-3dbffe56ff4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:43.331762 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-30145d20-cc51-40bf-93d6-3dbffe56ff4f None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:43.333979 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-30145d20-cc51-40bf-93d6-3dbffe56ff4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:43.345211 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1195/2379] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:39:43.349245 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43622ce-04e0-49e1-b337-df50fc2d08b2 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:43.360702 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43622ce-04e0-49e1-b337-df50fc2d08b2 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:43.365718 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954827fc-21da-4947-8055-babae70a9e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:43.366476 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954827fc-21da-4947-8055-babae70a9e71 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:43.366905 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954827fc-21da-4947-8055-babae70a9e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:43.367273 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954827fc-21da-4947-8055-babae70a9e71 admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:43.368088 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954827fc-21da-4947-8055-babae70a9e71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:39:43.368441 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954827fc-21da-4947-8055-babae70a9e71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:43.370067 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954827fc-21da-4947-8055-babae70a9e71 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:43.387785 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1185/2380] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:39:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:39:43.389928 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30145d20-cc51-40bf-93d6-3dbffe56ff4f None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:43.404469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30145d20-cc51-40bf-93d6-3dbffe56ff4f None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:43.409573 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7d3ff-b4b1-4620-b7e7-bb37ded39b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:43.410323 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7d3ff-b4b1-4620-b7e7-bb37ded39b82 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:43.410822 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7d3ff-b4b1-4620-b7e7-bb37ded39b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:43.411140 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7d3ff-b4b1-4620-b7e7-bb37ded39b82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:43.412073 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7d3ff-b4b1-4620-b7e7-bb37ded39b82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:43.414240 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7d3ff-b4b1-4620-b7e7-bb37ded39b82 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:43.451483 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1196/2381] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:39:43 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:39:43.453632 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954827fc-21da-4947-8055-babae70a9e71 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:43.465353 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954827fc-21da-4947-8055-babae70a9e71 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:43.537246 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3e148-f17b-406e-9de9-5ffdb5e0f42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:43.537922 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3e148-f17b-406e-9de9-5ffdb5e0f42c admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:43.538206 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3e148-f17b-406e-9de9-5ffdb5e0f42c admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:43.538487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3e148-f17b-406e-9de9-5ffdb5e0f42c admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:43.539188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3e148-f17b-406e-9de9-5ffdb5e0f42c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:43.541166 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3e148-f17b-406e-9de9-5ffdb5e0f42c admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:43.566753 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1186/2382] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:39:43 2022] POST /identity/v3/users => generated 286 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:39:43.574101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7d3ff-b4b1-4620-b7e7-bb37ded39b82 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:43.585394 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7d3ff-b4b1-4620-b7e7-bb37ded39b82 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:43.590181 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068d9b3-509b-47ee-9a88-5698438791f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:43.590855 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068d9b3-509b-47ee-9a88-5698438791f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:43.591151 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068d9b3-509b-47ee-9a88-5698438791f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:43.591420 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9068d9b3-509b-47ee-9a88-5698438791f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:43.592144 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9068d9b3-509b-47ee-9a88-5698438791f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:43.593774 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9068d9b3-509b-47ee-9a88-5698438791f2 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:43.605714 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1197/2383] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:39:43.610163 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3e148-f17b-406e-9de9-5ffdb5e0f42c admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:43.622257 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3e148-f17b-406e-9de9-5ffdb5e0f42c admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:43.627430 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc26d72-71bd-4522-ac67-393396b2ad25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:43.628119 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc26d72-71bd-4522-ac67-393396b2ad25 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:43.628458 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc26d72-71bd-4522-ac67-393396b2ad25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:43.628888 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc26d72-71bd-4522-ac67-393396b2ad25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:43.629806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc26d72-71bd-4522-ac67-393396b2ad25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:43.631552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc26d72-71bd-4522-ac67-393396b2ad25 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:43.643969 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1187/2384] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:39:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:39:43.648940 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068d9b3-509b-47ee-9a88-5698438791f2 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:43.659521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9068d9b3-509b-47ee-9a88-5698438791f2 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:43.664561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb229b-e967-4899-bb35-068cd1e12601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:43.665608 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb229b-e967-4899-bb35-068cd1e12601 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:43.665958 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb229b-e967-4899-bb35-068cd1e12601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:43.666323 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb229b-e967-4899-bb35-068cd1e12601 admin admin] PATH_INFO: `/v3/projects/2f49de7a803946489354eb3e4191b72b/users/703ab789aef14c33b71c5ffa794e5635/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:43.685051 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb229b-e967-4899-bb35-068cd1e12601 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f49de7a803946489354eb3e4191b72b, user_id=703ab789aef14c33b71c5ffa794e5635, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:43.687734 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb229b-e967-4899-bb35-068cd1e12601 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:43.704850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1198/2385] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:39:43 2022] PUT /identity/v3/projects/2f49de7a803946489354eb3e4191b72b/users/703ab789aef14c33b71c5ffa794e5635/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:43.716136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702e94a-86e5-4e74-862a-f7dc23b54e8f None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:43.716630 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702e94a-86e5-4e74-862a-f7dc23b54e8f None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:43.717008 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e702e94a-86e5-4e74-862a-f7dc23b54e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:43.769355 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-e702e94a-86e5-4e74-862a-f7dc23b54e8f None None] MFA Rules not processed for user `703ab789aef14c33b71c5ffa794e5635`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:43.771847 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-e702e94a-86e5-4e74-862a-f7dc23b54e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:43.844525 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:43.869532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1188/2386] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:39:43.881352 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb229b-e967-4899-bb35-068cd1e12601 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:43.885911 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-f5eb229b-e967-4899-bb35-068cd1e12601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:43.933563 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:43.959896 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb229b-e967-4899-bb35-068cd1e12601 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:43.963686 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704610e2-903e-4fbc-812b-5045983956f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:43.964381 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704610e2-903e-4fbc-812b-5045983956f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:43.964689 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704610e2-903e-4fbc-812b-5045983956f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:43.964965 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704610e2-903e-4fbc-812b-5045983956f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:43.974220 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704610e2-903e-4fbc-812b-5045983956f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:43.975787 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704610e2-903e-4fbc-812b-5045983956f8 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:43.984927 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1199/2387] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:39:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:39:44.006672 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702e94a-86e5-4e74-862a-f7dc23b54e8f None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:44.090352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e702e94a-86e5-4e74-862a-f7dc23b54e8f None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:44.095852 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ed81d-9d46-48d1-8972-e3e161864259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:44.096760 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ed81d-9d46-48d1-8972-e3e161864259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:44.097047 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ed81d-9d46-48d1-8972-e3e161864259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:44.097323 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ed81d-9d46-48d1-8972-e3e161864259 admin admin] PATH_INFO: `/v3/users/703ab789aef14c33b71c5ffa794e5635` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:44.099715 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ed81d-9d46-48d1-8972-e3e161864259 admin admin] RBAC: Authorizing `identity:delete_user(user_id=703ab789aef14c33b71c5ffa794e5635)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:44.101663 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ed81d-9d46-48d1-8972-e3e161864259 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:44.156988 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-1a0ed81d-9d46-48d1-8972-e3e161864259 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '703ab789aef14c33b71c5ffa794e5635'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:44.157640 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-1a0ed81d-9d46-48d1-8972-e3e161864259 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '703ab789aef14c33b71c5ffa794e5635'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:44.165816 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-1a0ed81d-9d46-48d1-8972-e3e161864259 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '703ab789aef14c33b71c5ffa794e5635'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:44.178521 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-1a0ed81d-9d46-48d1-8972-e3e161864259 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '703ab789aef14c33b71c5ffa794e5635'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:44.187993 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-1a0ed81d-9d46-48d1-8972-e3e161864259 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '703ab789aef14c33b71c5ffa794e5635'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:44.194653 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1189/2388] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:39:44 2022] DELETE /identity/v3/users/703ab789aef14c33b71c5ffa794e5635 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:44.199173 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd809f-cfc4-4f96-8e16-c3ca8e2afedb None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:44.199488 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd809f-cfc4-4f96-8e16-c3ca8e2afedb None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:44.199764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd809f-cfc4-4f96-8e16-c3ca8e2afedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:44.217900 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-48bd809f-cfc4-4f96-8e16-c3ca8e2afedb None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:44.219079 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-48bd809f-cfc4-4f96-8e16-c3ca8e2afedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:44.298768 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1200/2389] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:39:44.309559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ed81d-9d46-48d1-8972-e3e161864259 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:44.311267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-1a0ed81d-9d46-48d1-8972-e3e161864259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:44.402823 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ed81d-9d46-48d1-8972-e3e161864259 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:44.407950 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbfd60b-7b47-4292-a55a-0072ae08169a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:44.409612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbfd60b-7b47-4292-a55a-0072ae08169a service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:44.409976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbfd60b-7b47-4292-a55a-0072ae08169a service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:44.410328 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbfd60b-7b47-4292-a55a-0072ae08169a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:44.423838 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbfd60b-7b47-4292-a55a-0072ae08169a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:44.425607 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbfd60b-7b47-4292-a55a-0072ae08169a service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:44.441787 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1190/2390] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:39:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:39:44.798154 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd809f-cfc4-4f96-8e16-c3ca8e2afedb None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:44.799795 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-48bd809f-cfc4-4f96-8e16-c3ca8e2afedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:44.815941 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd809f-cfc4-4f96-8e16-c3ca8e2afedb None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:44.820935 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817a0ac-845e-4401-a132-3e8f6d6818ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:44.821801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817a0ac-845e-4401-a132-3e8f6d6818ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:44.822081 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817a0ac-845e-4401-a132-3e8f6d6818ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:44.822351 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817a0ac-845e-4401-a132-3e8f6d6818ff admin admin] PATH_INFO: `/v3/projects/2f49de7a803946489354eb3e4191b72b` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:44.823541 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5817a0ac-845e-4401-a132-3e8f6d6818ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f49de7a803946489354eb3e4191b72b)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:44.826630 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5817a0ac-845e-4401-a132-3e8f6d6818ff admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:44.880063 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5817a0ac-845e-4401-a132-3e8f6d6818ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f49de7a803946489354eb3e4191b72b', 'request_id': 'req-5817a0ac-845e-4401-a132-3e8f6d6818ff'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:44.886058 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5817a0ac-845e-4401-a132-3e8f6d6818ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f49de7a803946489354eb3e4191b72b', 'request_id': 'req-5817a0ac-845e-4401-a132-3e8f6d6818ff'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:44.893318 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5817a0ac-845e-4401-a132-3e8f6d6818ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f49de7a803946489354eb3e4191b72b', 'request_id': 'req-5817a0ac-845e-4401-a132-3e8f6d6818ff'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:44.906013 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5817a0ac-845e-4401-a132-3e8f6d6818ff admin admin] The token cache is being invalidate because project 2f49de7a803946489354eb3e4191b72b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:39:44.906315 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5817a0ac-845e-4401-a132-3e8f6d6818ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:44.912702 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5817a0ac-845e-4401-a132-3e8f6d6818ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:44.914450 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1201/2391] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:39:44 2022] DELETE /identity/v3/projects/2f49de7a803946489354eb3e4191b72b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:44.919830 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e251c54-b4ca-47f9-bc0b-875bc77ff07d None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:44.920126 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e251c54-b4ca-47f9-bc0b-875bc77ff07d None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:44.920587 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e251c54-b4ca-47f9-bc0b-875bc77ff07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:44.939329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-9e251c54-b4ca-47f9-bc0b-875bc77ff07d None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:44.940795 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-9e251c54-b4ca-47f9-bc0b-875bc77ff07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:44.945552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:45.017655 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1191/2392] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:39:45.026222 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a9bd6-5600-4a4a-9009-a4a83c035d50 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:45.027134 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a9bd6-5600-4a4a-9009-a4a83c035d50 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:45.027715 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a9bd6-5600-4a4a-9009-a4a83c035d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:45.058510 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-ba5a9bd6-5600-4a4a-9009-a4a83c035d50 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:45.061061 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ba5a9bd6-5600-4a4a-9009-a4a83c035d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:45.074363 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1202/2393] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:39:45.078940 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e251c54-b4ca-47f9-bc0b-875bc77ff07d None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:45.092877 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e251c54-b4ca-47f9-bc0b-875bc77ff07d None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:45.105186 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f9685c-e2bd-463a-85ae-08167f3a3a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:45.105929 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9685c-e2bd-463a-85ae-08167f3a3a8f admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:45.106477 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9685c-e2bd-463a-85ae-08167f3a3a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:45.107150 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9685c-e2bd-463a-85ae-08167f3a3a8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:45.108814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f9685c-e2bd-463a-85ae-08167f3a3a8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:39:45.109299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f9685c-e2bd-463a-85ae-08167f3a3a8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:45.111148 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f9685c-e2bd-463a-85ae-08167f3a3a8f admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:45.132737 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1192/2394] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:39:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:39:45.134313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a9bd6-5600-4a4a-9009-a4a83c035d50 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:45.145701 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a9bd6-5600-4a4a-9009-a4a83c035d50 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:45.150474 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a41b3-d422-4b2a-b90a-b15ad9b9efbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:45.151244 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a41b3-d422-4b2a-b90a-b15ad9b9efbd admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:45.151529 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a41b3-d422-4b2a-b90a-b15ad9b9efbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:45.151822 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a41b3-d422-4b2a-b90a-b15ad9b9efbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:45.153132 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a41b3-d422-4b2a-b90a-b15ad9b9efbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:45.155028 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91a41b3-d422-4b2a-b90a-b15ad9b9efbd admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:45.188578 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1203/2395] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:39:45 2022] POST /identity/v3/projects => generated 363 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:39:45.193737 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f9685c-e2bd-463a-85ae-08167f3a3a8f admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:45.204810 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f9685c-e2bd-463a-85ae-08167f3a3a8f admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:45.267625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61538a-b8f9-424a-a1b3-82db91eb88d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:45.268333 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61538a-b8f9-424a-a1b3-82db91eb88d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:45.268625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61538a-b8f9-424a-a1b3-82db91eb88d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:45.268902 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61538a-b8f9-424a-a1b3-82db91eb88d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:45.269598 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61538a-b8f9-424a-a1b3-82db91eb88d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:45.272125 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61538a-b8f9-424a-a1b3-82db91eb88d6 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:45.293312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1193/2396] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:39:45 2022] POST /identity/v3/users => generated 282 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:39:45.297470 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a41b3-d422-4b2a-b90a-b15ad9b9efbd admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:45.308074 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a41b3-d422-4b2a-b90a-b15ad9b9efbd admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:45.312792 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd60cd-b970-48d6-8f98-fa464450228d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:45.313444 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd60cd-b970-48d6-8f98-fa464450228d admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:45.313725 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd60cd-b970-48d6-8f98-fa464450228d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:45.314003 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd60cd-b970-48d6-8f98-fa464450228d admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:45.314713 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cd60cd-b970-48d6-8f98-fa464450228d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:45.316358 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cd60cd-b970-48d6-8f98-fa464450228d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:45.327672 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1204/2397] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:39:45.333065 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61538a-b8f9-424a-a1b3-82db91eb88d6 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:45.343312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61538a-b8f9-424a-a1b3-82db91eb88d6 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:45.349487 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f536bd7a-508b-42e1-8f39-b5ead98a658a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:45.350379 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536bd7a-508b-42e1-8f39-b5ead98a658a admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:45.350691 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536bd7a-508b-42e1-8f39-b5ead98a658a admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:45.350971 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f536bd7a-508b-42e1-8f39-b5ead98a658a admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:45.351827 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f536bd7a-508b-42e1-8f39-b5ead98a658a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:45.353718 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f536bd7a-508b-42e1-8f39-b5ead98a658a admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:45.365194 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1194/2398] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:39:45.369440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd60cd-b970-48d6-8f98-fa464450228d admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:45.382656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd60cd-b970-48d6-8f98-fa464450228d admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:45.388112 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c435db4-5dd0-4d33-87e7-db04a7e0e464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:45.389047 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c435db4-5dd0-4d33-87e7-db04a7e0e464 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:45.389331 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c435db4-5dd0-4d33-87e7-db04a7e0e464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:45.389604 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c435db4-5dd0-4d33-87e7-db04a7e0e464 admin admin] PATH_INFO: `/v3/projects/1adfc0db703244038041f32fc483ad55/users/bd8065d17fad415ab32d1fa2ce36180f/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:45.407587 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c435db4-5dd0-4d33-87e7-db04a7e0e464 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1adfc0db703244038041f32fc483ad55, user_id=bd8065d17fad415ab32d1fa2ce36180f, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:45.410642 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c435db4-5dd0-4d33-87e7-db04a7e0e464 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:45.413534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:45.426064 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1205/2399] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:39:45 2022] PUT /identity/v3/projects/1adfc0db703244038041f32fc483ad55/users/bd8065d17fad415ab32d1fa2ce36180f/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:45.437035 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3c74e-3f97-498a-a638-cd38d7027114 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:45.437326 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3c74e-3f97-498a-a638-cd38d7027114 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:45.437605 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3c74e-3f97-498a-a638-cd38d7027114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:45.465449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-6de3c74e-3f97-498a-a638-cd38d7027114 None None] MFA Rules not processed for user `bd8065d17fad415ab32d1fa2ce36180f`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:45.466661 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-6de3c74e-3f97-498a-a638-cd38d7027114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:45.541103 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1195/2400] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:45 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:39:45.552149 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c435db4-5dd0-4d33-87e7-db04a7e0e464 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:45.554025 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1c435db4-5dd0-4d33-87e7-db04a7e0e464 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:45.618666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c435db4-5dd0-4d33-87e7-db04a7e0e464 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:45.622341 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290ea79-ccf3-4eb4-8c49-c2cf1af7f146 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:45.622993 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290ea79-ccf3-4eb4-8c49-c2cf1af7f146 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:45.623274 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290ea79-ccf3-4eb4-8c49-c2cf1af7f146 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:45.623548 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b290ea79-ccf3-4eb4-8c49-c2cf1af7f146 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:45.633384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b290ea79-ccf3-4eb4-8c49-c2cf1af7f146 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:45.634808 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b290ea79-ccf3-4eb4-8c49-c2cf1af7f146 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:45.643899 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1206/2401] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:39:45 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:39:47.619758 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3c74e-3f97-498a-a638-cd38d7027114 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:47.621287 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-6de3c74e-3f97-498a-a638-cd38d7027114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:47.627537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:47.685890 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3c74e-3f97-498a-a638-cd38d7027114 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:47.689505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f3295-0e3f-4698-b11e-9817725decba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:47.690155 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f3295-0e3f-4698-b11e-9817725decba service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:47.690444 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f3295-0e3f-4698-b11e-9817725decba service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:47.690728 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f3295-0e3f-4698-b11e-9817725decba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:47.693186 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-6d9f3295-0e3f-4698-b11e-9817725decba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:47.758684 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f3295-0e3f-4698-b11e-9817725decba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:47.760456 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f3295-0e3f-4698-b11e-9817725decba service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:47.770598 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1196/2402] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:39:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:39:48.142636 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290ea79-ccf3-4eb4-8c49-c2cf1af7f146 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:48.144077 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b290ea79-ccf3-4eb4-8c49-c2cf1af7f146 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:48.156664 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b290ea79-ccf3-4eb4-8c49-c2cf1af7f146 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:48.160557 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed16cc7d-a3e7-4ce4-a56f-d2f603098684 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:48.161197 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16cc7d-a3e7-4ce4-a56f-d2f603098684 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:48.161483 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16cc7d-a3e7-4ce4-a56f-d2f603098684 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:48.161762 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16cc7d-a3e7-4ce4-a56f-d2f603098684 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:48.163608 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ed16cc7d-a3e7-4ce4-a56f-d2f603098684 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:48.175953 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed16cc7d-a3e7-4ce4-a56f-d2f603098684 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:48.178278 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed16cc7d-a3e7-4ce4-a56f-d2f603098684 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:48.188572 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1207/2403] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:39:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:39:51.632994 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f3295-0e3f-4698-b11e-9817725decba service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:51.634743 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-6d9f3295-0e3f-4698-b11e-9817725decba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:51.710900 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed16cc7d-a3e7-4ce4-a56f-d2f603098684 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:51.714029 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ed16cc7d-a3e7-4ce4-a56f-d2f603098684 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:51.741050 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed16cc7d-a3e7-4ce4-a56f-d2f603098684 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:51.768370 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd603bf1-06dc-4207-a2ec-cf2782d8e6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:51.771625 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd603bf1-06dc-4207-a2ec-cf2782d8e6ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:51.772559 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd603bf1-06dc-4207-a2ec-cf2782d8e6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:51.773509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd603bf1-06dc-4207-a2ec-cf2782d8e6ef admin admin] PATH_INFO: `/v3/users/ef222fe025534a9e942f8c5c4ed1e3e7` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:51.777559 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd603bf1-06dc-4207-a2ec-cf2782d8e6ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef222fe025534a9e942f8c5c4ed1e3e7)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:51.782410 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd603bf1-06dc-4207-a2ec-cf2782d8e6ef admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:51.862910 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f3295-0e3f-4698-b11e-9817725decba service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:51.871372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e328cf46-b0ec-4268-b5be-0c219171f4fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc0f137e0d24dd2a460689bc17a1b98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:51.872482 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328cf46-b0ec-4268-b5be-0c219171f4fd service placement] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:51.873905 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328cf46-b0ec-4268-b5be-0c219171f4fd service placement] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:51.874371 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328cf46-b0ec-4268-b5be-0c219171f4fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:51.876430 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-e328cf46-b0ec-4268-b5be-0c219171f4fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:51.903102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e328cf46-b0ec-4268-b5be-0c219171f4fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:51.914642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e328cf46-b0ec-4268-b5be-0c219171f4fd service placement] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:51.921184 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cd603bf1-06dc-4207-a2ec-cf2782d8e6ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef222fe025534a9e942f8c5c4ed1e3e7'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:51.939291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1197/2404] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:39:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:39:51.945668 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a90589-a735-43f6-868c-3431003be836 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:51.946015 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cd603bf1-06dc-4207-a2ec-cf2782d8e6ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef222fe025534a9e942f8c5c4ed1e3e7'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:51.946862 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a90589-a735-43f6-868c-3431003be836 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:51.947599 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a90589-a735-43f6-868c-3431003be836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:51.958533 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cd603bf1-06dc-4207-a2ec-cf2782d8e6ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef222fe025534a9e942f8c5c4ed1e3e7'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:51.960158 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cd603bf1-06dc-4207-a2ec-cf2782d8e6ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef222fe025534a9e942f8c5c4ed1e3e7'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:51.963614 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-cd603bf1-06dc-4207-a2ec-cf2782d8e6ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef222fe025534a9e942f8c5c4ed1e3e7'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:51.999321 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-a7a90589-a735-43f6-868c-3431003be836 None None] MFA Rules not processed for user `733f8296ec3d4795a1140c1e1cb6eb95`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:52.001531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-a7a90589-a735-43f6-868c-3431003be836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:52.006105 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1208/2405] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:39:51 2022] DELETE /identity/v3/users/ef222fe025534a9e942f8c5c4ed1e3e7 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:39:52.084137 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:52.102596 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1198/2406] 10.222.0.16 () {58 vars in 1009 bytes} [Mon May 23 12:39:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:39:52.163135 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd603bf1-06dc-4207-a2ec-cf2782d8e6ef admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:52.164660 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-cd603bf1-06dc-4207-a2ec-cf2782d8e6ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:52.242132 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:52.270977 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd603bf1-06dc-4207-a2ec-cf2782d8e6ef admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:52.277931 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9a1a8-d05d-4bd6-86b3-d0c5e9147c72 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71fb9a79ac494635804b93d4a0048205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:52.278601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9a1a8-d05d-4bd6-86b3-d0c5e9147c72 service swift] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:52.278883 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9a1a8-d05d-4bd6-86b3-d0c5e9147c72 service swift] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:52.279163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9a1a8-d05d-4bd6-86b3-d0c5e9147c72 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:52.288552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da9a1a8-d05d-4bd6-86b3-d0c5e9147c72 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:52.290563 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da9a1a8-d05d-4bd6-86b3-d0c5e9147c72 service swift] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:52.302789 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1209/2407] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 12:39:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 12:39:52.356217 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a90589-a735-43f6-868c-3431003be836 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:52.358193 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-a7a90589-a735-43f6-868c-3431003be836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:52.489507 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a90589-a735-43f6-868c-3431003be836 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:52.496515 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974e86b-1f0b-4858-901e-f438d47cf64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:52.498134 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974e86b-1f0b-4858-901e-f438d47cf64e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:52.498492 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974e86b-1f0b-4858-901e-f438d47cf64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:52.498878 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974e86b-1f0b-4858-901e-f438d47cf64e admin admin] PATH_INFO: `/v3/projects/da82f9c566664cb1a974cf0d1bf1b09d` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:52.510841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2974e86b-1f0b-4858-901e-f438d47cf64e admin admin] RBAC: Authorizing `identity:delete_project(project_id=da82f9c566664cb1a974cf0d1bf1b09d)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:52.513216 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2974e86b-1f0b-4858-901e-f438d47cf64e admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:52.606636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2974e86b-1f0b-4858-901e-f438d47cf64e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da82f9c566664cb1a974cf0d1bf1b09d', 'request_id': 'req-2974e86b-1f0b-4858-901e-f438d47cf64e'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:52.622486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2974e86b-1f0b-4858-901e-f438d47cf64e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da82f9c566664cb1a974cf0d1bf1b09d', 'request_id': 'req-2974e86b-1f0b-4858-901e-f438d47cf64e'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:52.634677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2974e86b-1f0b-4858-901e-f438d47cf64e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da82f9c566664cb1a974cf0d1bf1b09d', 'request_id': 'req-2974e86b-1f0b-4858-901e-f438d47cf64e'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:52.656049 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2974e86b-1f0b-4858-901e-f438d47cf64e admin admin] The token cache is being invalidate because project da82f9c566664cb1a974cf0d1bf1b09d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:39:52.656735 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2974e86b-1f0b-4858-901e-f438d47cf64e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:52.657702 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2974e86b-1f0b-4858-901e-f438d47cf64e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:52.664138 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1199/2408] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:39:52 2022] DELETE /identity/v3/projects/da82f9c566664cb1a974cf0d1bf1b09d => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:52.665457 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d62b1c-a307-474d-b14c-087d4570632d None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:52.665742 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d62b1c-a307-474d-b14c-087d4570632d None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:52.666029 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d62b1c-a307-474d-b14c-087d4570632d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:52.705873 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-b7d62b1c-a307-474d-b14c-087d4570632d None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:52.707410 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b7d62b1c-a307-474d-b14c-087d4570632d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:52.794058 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1210/2409] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:39:52.802968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb221ae-6ae0-4856-abee-242cbd7457d3 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:52.803530 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb221ae-6ae0-4856-abee-242cbd7457d3 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:52.804627 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb221ae-6ae0-4856-abee-242cbd7457d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:52.838010 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-6fb221ae-6ae0-4856-abee-242cbd7457d3 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:52.839804 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-6fb221ae-6ae0-4856-abee-242cbd7457d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:52.862239 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1200/2410] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:39:52.865052 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d62b1c-a307-474d-b14c-087d4570632d None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:52.887840 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d62b1c-a307-474d-b14c-087d4570632d None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:52.901014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81d981-47d2-4077-a14c-c49670684229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:52.902777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81d981-47d2-4077-a14c-c49670684229 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:52.906108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81d981-47d2-4077-a14c-c49670684229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:52.906399 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81d981-47d2-4077-a14c-c49670684229 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:52.911144 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81d981-47d2-4077-a14c-c49670684229 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:39:52.911580 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81d981-47d2-4077-a14c-c49670684229 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:52.916104 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81d981-47d2-4077-a14c-c49670684229 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:52.957653 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1211/2411] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:39:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:39:52.962321 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb221ae-6ae0-4856-abee-242cbd7457d3 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:52.975315 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb221ae-6ae0-4856-abee-242cbd7457d3 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:52.989630 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d3671-ca0c-443e-9d2d-00b842611062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:52.991368 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d3671-ca0c-443e-9d2d-00b842611062 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:52.994251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d3671-ca0c-443e-9d2d-00b842611062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:52.995091 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d3671-ca0c-443e-9d2d-00b842611062 admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:52.997121 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d3671-ca0c-443e-9d2d-00b842611062 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:53.000395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d3671-ca0c-443e-9d2d-00b842611062 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:53.044750 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81d981-47d2-4077-a14c-c49670684229 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:53.046923 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3f81d981-47d2-4077-a14c-c49670684229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:53.074224 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81d981-47d2-4077-a14c-c49670684229 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:53.084131 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1201/2412] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:39:52 2022] POST /identity/v3/projects => generated 371 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:39:53.085497 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d3671-ca0c-443e-9d2d-00b842611062 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:53.111697 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29b91b-5d0f-4ca5-83c6-a7f50b9a00ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:53.115742 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29b91b-5d0f-4ca5-83c6-a7f50b9a00ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:53.116270 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29b91b-5d0f-4ca5-83c6-a7f50b9a00ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:53.116270 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29b91b-5d0f-4ca5-83c6-a7f50b9a00ab admin admin] PATH_INFO: `/v3/users/771d3e22beff44f7bbc3103fd75a35b4` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:53.121217 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29b91b-5d0f-4ca5-83c6-a7f50b9a00ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=771d3e22beff44f7bbc3103fd75a35b4)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:53.123201 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29b91b-5d0f-4ca5-83c6-a7f50b9a00ab admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:53.173872 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:53.200662 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d3671-ca0c-443e-9d2d-00b842611062 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:53.213119 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5122b-bf4a-41e7-af4d-bcd61a0af4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:53.214470 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5122b-bf4a-41e7-af4d-bcd61a0af4d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:53.215419 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5122b-bf4a-41e7-af4d-bcd61a0af4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:53.215790 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5122b-bf4a-41e7-af4d-bcd61a0af4d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:53.217505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d5122b-bf4a-41e7-af4d-bcd61a0af4d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:53.219612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d5122b-bf4a-41e7-af4d-bcd61a0af4d7 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:53.242168 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3a29b91b-5d0f-4ca5-83c6-a7f50b9a00ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '771d3e22beff44f7bbc3103fd75a35b4'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:53.259955 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1202/2413] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:39:53 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:39:53.268743 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5122b-bf4a-41e7-af4d-bcd61a0af4d7 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:53.279038 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3a29b91b-5d0f-4ca5-83c6-a7f50b9a00ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '771d3e22beff44f7bbc3103fd75a35b4'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:53.290761 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5122b-bf4a-41e7-af4d-bcd61a0af4d7 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:53.298574 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3a29b91b-5d0f-4ca5-83c6-a7f50b9a00ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '771d3e22beff44f7bbc3103fd75a35b4'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:53.300865 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3a29b91b-5d0f-4ca5-83c6-a7f50b9a00ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '771d3e22beff44f7bbc3103fd75a35b4'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:53.302413 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3a29b91b-5d0f-4ca5-83c6-a7f50b9a00ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '771d3e22beff44f7bbc3103fd75a35b4'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:53.309395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3cdf00-aaf3-4571-8c1e-5247b5b07f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:53.310728 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3cdf00-aaf3-4571-8c1e-5247b5b07f83 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:53.311755 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3cdf00-aaf3-4571-8c1e-5247b5b07f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:53.312053 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3cdf00-aaf3-4571-8c1e-5247b5b07f83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:53.312837 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3cdf00-aaf3-4571-8c1e-5247b5b07f83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:53.314817 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3cdf00-aaf3-4571-8c1e-5247b5b07f83 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:53.328972 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1212/2414] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:39:53 2022] DELETE /identity/v3/users/771d3e22beff44f7bbc3103fd75a35b4 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:53.348916 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1203/2415] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:39:53.352746 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29b91b-5d0f-4ca5-83c6-a7f50b9a00ab admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:53.354591 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3a29b91b-5d0f-4ca5-83c6-a7f50b9a00ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:53.445459 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:53.481046 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29b91b-5d0f-4ca5-83c6-a7f50b9a00ab admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:53.489069 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5568cdcd-352f-4dd3-87b7-d5c56824f75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:53.490281 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5568cdcd-352f-4dd3-87b7-d5c56824f75d admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:53.490795 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5568cdcd-352f-4dd3-87b7-d5c56824f75d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:53.491286 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5568cdcd-352f-4dd3-87b7-d5c56824f75d admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:53.492228 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5568cdcd-352f-4dd3-87b7-d5c56824f75d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:53.494030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5568cdcd-352f-4dd3-87b7-d5c56824f75d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:53.506760 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1213/2416] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:39:53.511632 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3cdf00-aaf3-4571-8c1e-5247b5b07f83 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:53.525143 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3cdf00-aaf3-4571-8c1e-5247b5b07f83 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:53.531986 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6e26d-03fc-410f-bf1b-c786f7a39628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:53.533263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6e26d-03fc-410f-bf1b-c786f7a39628 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:53.533780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6e26d-03fc-410f-bf1b-c786f7a39628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:53.534139 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6e26d-03fc-410f-bf1b-c786f7a39628 admin admin] PATH_INFO: `/v3/projects/2e8f6cf2d2aa4e9d953d19d298212304/users/93b9d29fb29540b0bee63abb1e68d9a2/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:53.553760 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6e26d-03fc-410f-bf1b-c786f7a39628 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e8f6cf2d2aa4e9d953d19d298212304, user_id=93b9d29fb29540b0bee63abb1e68d9a2, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:53.557143 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6e26d-03fc-410f-bf1b-c786f7a39628 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:53.582890 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1204/2417] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:39:53 2022] PUT /identity/v3/projects/2e8f6cf2d2aa4e9d953d19d298212304/users/93b9d29fb29540b0bee63abb1e68d9a2/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:53.593970 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b3ae2-1dcc-4298-8b6d-896bb8ab01e2 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:53.594338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b3ae2-1dcc-4298-8b6d-896bb8ab01e2 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:53.594569 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b3ae2-1dcc-4298-8b6d-896bb8ab01e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:53.605681 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6e26d-03fc-410f-bf1b-c786f7a39628 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:53.609618 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-39c6e26d-03fc-410f-bf1b-c786f7a39628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:53.651568 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-261b3ae2-1dcc-4298-8b6d-896bb8ab01e2 None None] MFA Rules not processed for user `93b9d29fb29540b0bee63abb1e68d9a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:53.654494 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-261b3ae2-1dcc-4298-8b6d-896bb8ab01e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:53.707170 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6e26d-03fc-410f-bf1b-c786f7a39628 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:53.717851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22efd54-26ca-48c1-bee0-f7ead29dab20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:53.718743 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22efd54-26ca-48c1-bee0-f7ead29dab20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:53.719030 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22efd54-26ca-48c1-bee0-f7ead29dab20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:53.719308 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22efd54-26ca-48c1-bee0-f7ead29dab20 admin admin] PATH_INFO: `/v3/projects/736376830ed340b2aa62c455c134c4e2` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:53.720629 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22efd54-26ca-48c1-bee0-f7ead29dab20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=736376830ed340b2aa62c455c134c4e2)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:53.722616 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22efd54-26ca-48c1-bee0-f7ead29dab20 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:53.753179 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1214/2418] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:39:53.760990 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b3ae2-1dcc-4298-8b6d-896bb8ab01e2 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:53.781479 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b3ae2-1dcc-4298-8b6d-896bb8ab01e2 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:53.797283 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-d22efd54-26ca-48c1-bee0-f7ead29dab20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '736376830ed340b2aa62c455c134c4e2', 'request_id': 'req-d22efd54-26ca-48c1-bee0-f7ead29dab20'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:53.811112 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-d22efd54-26ca-48c1-bee0-f7ead29dab20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '736376830ed340b2aa62c455c134c4e2', 'request_id': 'req-d22efd54-26ca-48c1-bee0-f7ead29dab20'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:53.822015 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-d22efd54-26ca-48c1-bee0-f7ead29dab20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '736376830ed340b2aa62c455c134c4e2', 'request_id': 'req-d22efd54-26ca-48c1-bee0-f7ead29dab20'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:53.839277 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-d22efd54-26ca-48c1-bee0-f7ead29dab20 admin admin] The token cache is being invalidate because project 736376830ed340b2aa62c455c134c4e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:39:53.839674 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-d22efd54-26ca-48c1-bee0-f7ead29dab20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:53.841169 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-d22efd54-26ca-48c1-bee0-f7ead29dab20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:53.843211 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1205/2419] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:39:53 2022] DELETE /identity/v3/projects/736376830ed340b2aa62c455c134c4e2 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:53.848641 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c581e3-836a-4c29-aa95-b124537d616c None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:53.849052 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c581e3-836a-4c29-aa95-b124537d616c None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:53.849407 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c581e3-836a-4c29-aa95-b124537d616c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:53.869689 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-f9c581e3-836a-4c29-aa95-b124537d616c None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:53.871015 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-f9c581e3-836a-4c29-aa95-b124537d616c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:53.884104 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b3760-0ccf-4235-a2f2-13e783c2b93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:53.884838 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b3760-0ccf-4235-a2f2-13e783c2b93e admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:53.885647 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b3760-0ccf-4235-a2f2-13e783c2b93e admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:53.885955 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b3760-0ccf-4235-a2f2-13e783c2b93e admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:53.886824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692b3760-0ccf-4235-a2f2-13e783c2b93e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:53.890540 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692b3760-0ccf-4235-a2f2-13e783c2b93e admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:53.913039 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:53.938564 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1215/2420] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:39:53 2022] POST /identity/v3/projects => generated 371 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:39:53.942927 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b3760-0ccf-4235-a2f2-13e783c2b93e admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:53.945097 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-692b3760-0ccf-4235-a2f2-13e783c2b93e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:53.949419 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1206/2421] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:39:53.962532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac77aa-84c2-4cae-8951-516e8f38786b None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:53.963016 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac77aa-84c2-4cae-8951-516e8f38786b None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:53.963310 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac77aa-84c2-4cae-8951-516e8f38786b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:53.998514 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-aaac77aa-84c2-4cae-8951-516e8f38786b None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:54.000504 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-aaac77aa-84c2-4cae-8951-516e8f38786b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:54.038572 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:54.044711 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b3760-0ccf-4235-a2f2-13e783c2b93e admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:54.052945 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0a6a5-6c46-4c39-9442-240d36b4613e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:54.053633 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0a6a5-6c46-4c39-9442-240d36b4613e admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:54.053924 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0a6a5-6c46-4c39-9442-240d36b4613e admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:54.054202 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0a6a5-6c46-4c39-9442-240d36b4613e admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:54.054901 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d0a6a5-6c46-4c39-9442-240d36b4613e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:54.056918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d0a6a5-6c46-4c39-9442-240d36b4613e admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:54.084780 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1216/2422] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:39:53 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:39:54.086470 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1207/2423] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:39:54.090456 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0a6a5-6c46-4c39-9442-240d36b4613e admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:54.091781 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac77aa-84c2-4cae-8951-516e8f38786b None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:54.102600 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac77aa-84c2-4cae-8951-516e8f38786b None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:54.107452 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252731a-da68-40b0-9f12-bfedccb1b2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:54.108087 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252731a-da68-40b0-9f12-bfedccb1b2e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:54.108509 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252731a-da68-40b0-9f12-bfedccb1b2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:54.108688 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252731a-da68-40b0-9f12-bfedccb1b2e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:54.109380 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252731a-da68-40b0-9f12-bfedccb1b2e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:39:54.109847 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252731a-da68-40b0-9f12-bfedccb1b2e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:54.111565 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252731a-da68-40b0-9f12-bfedccb1b2e4 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:54.112727 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0a6a5-6c46-4c39-9442-240d36b4613e admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:54.119584 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7e6ad-8d67-4f74-a4be-c2ebc6600102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:54.121037 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7e6ad-8d67-4f74-a4be-c2ebc6600102 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:54.121917 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7e6ad-8d67-4f74-a4be-c2ebc6600102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:54.122208 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7e6ad-8d67-4f74-a4be-c2ebc6600102 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:54.124343 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c7e6ad-8d67-4f74-a4be-c2ebc6600102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:54.130619 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c7e6ad-8d67-4f74-a4be-c2ebc6600102 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:54.130829 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1208/2424] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:39:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:39:54.133014 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252731a-da68-40b0-9f12-bfedccb1b2e4 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:54.142288 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1217/2425] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:39:54.147106 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252731a-da68-40b0-9f12-bfedccb1b2e4 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:54.147967 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7e6ad-8d67-4f74-a4be-c2ebc6600102 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:54.152032 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a3b46-f28b-49ad-983f-ae01c7f320e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:54.152737 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a3b46-f28b-49ad-983f-ae01c7f320e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:54.153028 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a3b46-f28b-49ad-983f-ae01c7f320e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:54.153316 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a3b46-f28b-49ad-983f-ae01c7f320e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:54.154016 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a3b46-f28b-49ad-983f-ae01c7f320e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:54.157265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a3b46-f28b-49ad-983f-ae01c7f320e3 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:54.166150 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7e6ad-8d67-4f74-a4be-c2ebc6600102 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:54.172549 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77a5df-3885-4405-b199-acdccf42d6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:54.173963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77a5df-3885-4405-b199-acdccf42d6c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:54.174280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77a5df-3885-4405-b199-acdccf42d6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:54.174564 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77a5df-3885-4405-b199-acdccf42d6c8 admin admin] PATH_INFO: `/v3/projects/624bfdf95f98436b908e1a13a228d03e/users/54c235b93daf4f049d709d7dc7092dc9/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:54.193200 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77a5df-3885-4405-b199-acdccf42d6c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=624bfdf95f98436b908e1a13a228d03e, user_id=54c235b93daf4f049d709d7dc7092dc9, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:54.197071 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77a5df-3885-4405-b199-acdccf42d6c8 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:54.204965 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1209/2426] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:39:54 2022] POST /identity/v3/projects => generated 371 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:39:54.206370 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a3b46-f28b-49ad-983f-ae01c7f320e3 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:54.226594 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1218/2427] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:39:54 2022] PUT /identity/v3/projects/624bfdf95f98436b908e1a13a228d03e/users/54c235b93daf4f049d709d7dc7092dc9/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:54.228155 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77a5df-3885-4405-b199-acdccf42d6c8 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:54.327051 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77a5df-3885-4405-b199-acdccf42d6c8 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:54.331196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a3b46-f28b-49ad-983f-ae01c7f320e3 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:54.334523 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c8f00-b489-410f-b11b-75b820eba8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:54.335489 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c8f00-b489-410f-b11b-75b820eba8ca admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:54.336125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c8f00-b489-410f-b11b-75b820eba8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:54.336636 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c8f00-b489-410f-b11b-75b820eba8ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:54.337795 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c8f00-b489-410f-b11b-75b820eba8ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:54.339849 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c8f00-b489-410f-b11b-75b820eba8ca admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:54.354644 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a35c89-d3a2-4dc7-99b5-a9866ad171d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:54.354644 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a35c89-d3a2-4dc7-99b5-a9866ad171d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:54.354644 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a35c89-d3a2-4dc7-99b5-a9866ad171d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:54.354644 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a35c89-d3a2-4dc7-99b5-a9866ad171d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:54.355120 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a35c89-d3a2-4dc7-99b5-a9866ad171d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:54.355120 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a35c89-d3a2-4dc7-99b5-a9866ad171d3 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:54.360906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1219/2428] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:39:54.365600 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c8f00-b489-410f-b11b-75b820eba8ca admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:54.379684 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c8f00-b489-410f-b11b-75b820eba8ca admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:54.380696 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1210/2429] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:39:54 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:39:54.384711 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a35c89-d3a2-4dc7-99b5-a9866ad171d3 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:54.388703 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47723dc-0bb2-42e8-998d-aec4f4bbfe44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:54.389835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47723dc-0bb2-42e8-998d-aec4f4bbfe44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:54.390277 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47723dc-0bb2-42e8-998d-aec4f4bbfe44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:54.390851 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47723dc-0bb2-42e8-998d-aec4f4bbfe44 admin admin] PATH_INFO: `/v3/projects/624bfdf95f98436b908e1a13a228d03e/users/54c235b93daf4f049d709d7dc7092dc9/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:54.398403 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a35c89-d3a2-4dc7-99b5-a9866ad171d3 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:54.402044 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47723dc-0bb2-42e8-998d-aec4f4bbfe44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=624bfdf95f98436b908e1a13a228d03e, user_id=54c235b93daf4f049d709d7dc7092dc9, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:54.406108 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e03f1c-4ed0-4983-8342-3e28dce019ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:54.406999 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e03f1c-4ed0-4983-8342-3e28dce019ae admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:54.407300 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e03f1c-4ed0-4983-8342-3e28dce019ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:54.407588 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e03f1c-4ed0-4983-8342-3e28dce019ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:54.408368 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e03f1c-4ed0-4983-8342-3e28dce019ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:54.409997 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e03f1c-4ed0-4983-8342-3e28dce019ae admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:54.411174 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47723dc-0bb2-42e8-998d-aec4f4bbfe44 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:54.425054 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1211/2430] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:39:54.430957 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e03f1c-4ed0-4983-8342-3e28dce019ae admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:54.434629 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1220/2431] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:39:54 2022] PUT /identity/v3/projects/624bfdf95f98436b908e1a13a228d03e/users/54c235b93daf4f049d709d7dc7092dc9/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:54.447667 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e03f1c-4ed0-4983-8342-3e28dce019ae admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:54.451072 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abbfc9-ab52-434f-a0f3-73bef6f6afc3 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:54.451439 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abbfc9-ab52-434f-a0f3-73bef6f6afc3 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:54.451931 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abbfc9-ab52-434f-a0f3-73bef6f6afc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:54.475911 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:54.516211 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-05abbfc9-ab52-434f-a0f3-73bef6f6afc3 None None] MFA Rules not processed for user `54c235b93daf4f049d709d7dc7092dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:54.517584 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-05abbfc9-ab52-434f-a0f3-73bef6f6afc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:54.550240 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70028784-7527-4c16-9058-ebdaaba2a5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:54.551651 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70028784-7527-4c16-9058-ebdaaba2a5b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:54.551942 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70028784-7527-4c16-9058-ebdaaba2a5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:54.552266 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70028784-7527-4c16-9058-ebdaaba2a5b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:54.553355 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70028784-7527-4c16-9058-ebdaaba2a5b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:54.554990 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70028784-7527-4c16-9058-ebdaaba2a5b2 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:54.576695 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1212/2432] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:39:54.581308 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70028784-7527-4c16-9058-ebdaaba2a5b2 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:54.603713 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70028784-7527-4c16-9058-ebdaaba2a5b2 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:54.609386 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3fc028-28ae-49b0-8cfa-7ee415be398d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:54.610298 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3fc028-28ae-49b0-8cfa-7ee415be398d admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:54.610575 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3fc028-28ae-49b0-8cfa-7ee415be398d admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:54.610993 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3fc028-28ae-49b0-8cfa-7ee415be398d admin admin] PATH_INFO: `/v3/projects/3405fd7245fc467ea1938f9251662f0d/users/7ea2ed0a883248948c47f188c54bd42a/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:54.640702 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1221/2433] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:54 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:39:54.640891 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3fc028-28ae-49b0-8cfa-7ee415be398d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3405fd7245fc467ea1938f9251662f0d, user_id=7ea2ed0a883248948c47f188c54bd42a, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:54.642311 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3fc028-28ae-49b0-8cfa-7ee415be398d admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:54.650085 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abbfc9-ab52-434f-a0f3-73bef6f6afc3 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:54.651549 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-05abbfc9-ab52-434f-a0f3-73bef6f6afc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:54.671264 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1213/2434] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:39:54 2022] PUT /identity/v3/projects/3405fd7245fc467ea1938f9251662f0d/users/7ea2ed0a883248948c47f188c54bd42a/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:54.682552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18008b-3933-4861-a1b9-60b5a02d1e55 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:54.683007 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18008b-3933-4861-a1b9-60b5a02d1e55 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:54.684524 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18008b-3933-4861-a1b9-60b5a02d1e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:54.689680 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:54.717848 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-2c18008b-3933-4861-a1b9-60b5a02d1e55 None None] MFA Rules not processed for user `7ea2ed0a883248948c47f188c54bd42a`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:54.719674 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-2c18008b-3933-4861-a1b9-60b5a02d1e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:54.742105 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abbfc9-ab52-434f-a0f3-73bef6f6afc3 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:54.794918 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1214/2435] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:39:54.808659 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18008b-3933-4861-a1b9-60b5a02d1e55 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:54.810368 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-2c18008b-3933-4861-a1b9-60b5a02d1e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:54.821720 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ec2ab-3040-445a-84c0-9ee90b8f4b20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:54.822643 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ec2ab-3040-445a-84c0-9ee90b8f4b20 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:54.823262 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ec2ab-3040-445a-84c0-9ee90b8f4b20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:54.823547 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92ec2ab-3040-445a-84c0-9ee90b8f4b20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:54.904314 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ec2ab-3040-445a-84c0-9ee90b8f4b20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:54.906548 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92ec2ab-3040-445a-84c0-9ee90b8f4b20 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:54.917180 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18008b-3933-4861-a1b9-60b5a02d1e55 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:54.924275 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4acafe-c7fd-4aee-8ca6-02c48963412f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:54.925060 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4acafe-c7fd-4aee-8ca6-02c48963412f service nova] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:54.925470 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4acafe-c7fd-4aee-8ca6-02c48963412f service nova] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:54.925765 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4acafe-c7fd-4aee-8ca6-02c48963412f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:54.935708 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4acafe-c7fd-4aee-8ca6-02c48963412f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:54.937624 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4acafe-c7fd-4aee-8ca6-02c48963412f service nova] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:54.937804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1222/2436] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:39:54 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:39:54.953115 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1215/2437] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:39:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:39:55.471341 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ec2ab-3040-445a-84c0-9ee90b8f4b20 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:55.555289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92ec2ab-3040-445a-84c0-9ee90b8f4b20 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:55.561426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7326f7-5fa8-456d-a27f-6f9ddcfb2fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:55.562454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7326f7-5fa8-456d-a27f-6f9ddcfb2fb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:55.562783 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7326f7-5fa8-456d-a27f-6f9ddcfb2fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:55.563094 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7326f7-5fa8-456d-a27f-6f9ddcfb2fb0 admin admin] PATH_INFO: `/v3/users/93b9d29fb29540b0bee63abb1e68d9a2` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:55.564578 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7326f7-5fa8-456d-a27f-6f9ddcfb2fb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93b9d29fb29540b0bee63abb1e68d9a2)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:55.567097 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7326f7-5fa8-456d-a27f-6f9ddcfb2fb0 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:55.604311 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:55.640888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-af7326f7-5fa8-456d-a27f-6f9ddcfb2fb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93b9d29fb29540b0bee63abb1e68d9a2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:55.663807 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-af7326f7-5fa8-456d-a27f-6f9ddcfb2fb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93b9d29fb29540b0bee63abb1e68d9a2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:55.673970 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-af7326f7-5fa8-456d-a27f-6f9ddcfb2fb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93b9d29fb29540b0bee63abb1e68d9a2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:55.675090 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-af7326f7-5fa8-456d-a27f-6f9ddcfb2fb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93b9d29fb29540b0bee63abb1e68d9a2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:55.676402 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-af7326f7-5fa8-456d-a27f-6f9ddcfb2fb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93b9d29fb29540b0bee63abb1e68d9a2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:55.703386 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1223/2438] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:39:55 2022] DELETE /identity/v3/users/93b9d29fb29540b0bee63abb1e68d9a2 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:55.711181 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da15e9-58dc-4b35-8deb-d702e295a768 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:55.711673 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da15e9-58dc-4b35-8deb-d702e295a768 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:55.712186 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da15e9-58dc-4b35-8deb-d702e295a768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:55.731990 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:55.743179 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-f5da15e9-58dc-4b35-8deb-d702e295a768 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:55.749865 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-f5da15e9-58dc-4b35-8deb-d702e295a768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:55.861553 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1216/2439] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:39:55.871572 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7326f7-5fa8-456d-a27f-6f9ddcfb2fb0 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:55.873097 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-af7326f7-5fa8-456d-a27f-6f9ddcfb2fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:55.959451 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7326f7-5fa8-456d-a27f-6f9ddcfb2fb0 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:55.963400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15afb37-1ca9-4b1e-ad98-6bc80c26b202 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:55.964249 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15afb37-1ca9-4b1e-ad98-6bc80c26b202 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:55.964619 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15afb37-1ca9-4b1e-ad98-6bc80c26b202 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:55.964958 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15afb37-1ca9-4b1e-ad98-6bc80c26b202 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:55.978206 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15afb37-1ca9-4b1e-ad98-6bc80c26b202 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:55.980462 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15afb37-1ca9-4b1e-ad98-6bc80c26b202 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:55.991819 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1224/2440] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:39:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:39:56.333924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da15e9-58dc-4b35-8deb-d702e295a768 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:56.335554 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-f5da15e9-58dc-4b35-8deb-d702e295a768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:56.354757 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da15e9-58dc-4b35-8deb-d702e295a768 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:56.359656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235147a6-d271-4fbe-b0e3-e841159e659f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:56.360583 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235147a6-d271-4fbe-b0e3-e841159e659f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:56.361418 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235147a6-d271-4fbe-b0e3-e841159e659f admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:56.361737 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235147a6-d271-4fbe-b0e3-e841159e659f admin admin] PATH_INFO: `/v3/projects/2e8f6cf2d2aa4e9d953d19d298212304` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:56.363444 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235147a6-d271-4fbe-b0e3-e841159e659f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e8f6cf2d2aa4e9d953d19d298212304)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:56.365698 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235147a6-d271-4fbe-b0e3-e841159e659f admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:56.429002 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-235147a6-d271-4fbe-b0e3-e841159e659f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e8f6cf2d2aa4e9d953d19d298212304', 'request_id': 'req-235147a6-d271-4fbe-b0e3-e841159e659f'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:56.441326 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-235147a6-d271-4fbe-b0e3-e841159e659f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e8f6cf2d2aa4e9d953d19d298212304', 'request_id': 'req-235147a6-d271-4fbe-b0e3-e841159e659f'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:56.451165 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-235147a6-d271-4fbe-b0e3-e841159e659f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e8f6cf2d2aa4e9d953d19d298212304', 'request_id': 'req-235147a6-d271-4fbe-b0e3-e841159e659f'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:56.482990 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-235147a6-d271-4fbe-b0e3-e841159e659f admin admin] The token cache is being invalidate because project 2e8f6cf2d2aa4e9d953d19d298212304 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:39:56.483400 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-235147a6-d271-4fbe-b0e3-e841159e659f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:56.484141 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-235147a6-d271-4fbe-b0e3-e841159e659f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:56.486128 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1217/2441] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:39:56 2022] DELETE /identity/v3/projects/2e8f6cf2d2aa4e9d953d19d298212304 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:56.490688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15afb37-1ca9-4b1e-ad98-6bc80c26b202 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:56.504739 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15afb37-1ca9-4b1e-ad98-6bc80c26b202 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:56.617526 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafbe86d-c338-40b6-a3fc-2bef0acb39e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:56.618970 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafbe86d-c338-40b6-a3fc-2bef0acb39e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:56.619504 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafbe86d-c338-40b6-a3fc-2bef0acb39e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:56.619919 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafbe86d-c338-40b6-a3fc-2bef0acb39e0 admin admin] PATH_INFO: `/v3/users/54c235b93daf4f049d709d7dc7092dc9` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:56.621304 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafbe86d-c338-40b6-a3fc-2bef0acb39e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54c235b93daf4f049d709d7dc7092dc9)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:56.623275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafbe86d-c338-40b6-a3fc-2bef0acb39e0 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:56.690520 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-fafbe86d-c338-40b6-a3fc-2bef0acb39e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54c235b93daf4f049d709d7dc7092dc9'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:56.692760 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:56.714144 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-fafbe86d-c338-40b6-a3fc-2bef0acb39e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54c235b93daf4f049d709d7dc7092dc9'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:56.725350 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-fafbe86d-c338-40b6-a3fc-2bef0acb39e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54c235b93daf4f049d709d7dc7092dc9'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:56.726266 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-fafbe86d-c338-40b6-a3fc-2bef0acb39e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54c235b93daf4f049d709d7dc7092dc9'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:56.727093 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-fafbe86d-c338-40b6-a3fc-2bef0acb39e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54c235b93daf4f049d709d7dc7092dc9'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:56.747753 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1225/2442] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:39:56 2022] DELETE /identity/v3/users/54c235b93daf4f049d709d7dc7092dc9 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:57.187660 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235147a6-d271-4fbe-b0e3-e841159e659f admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:57.189558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-235147a6-d271-4fbe-b0e3-e841159e659f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:57.232065 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235147a6-d271-4fbe-b0e3-e841159e659f admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:57.314722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a57641-029f-41be-b3fa-143d3a5fadf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:57.315677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a57641-029f-41be-b3fa-143d3a5fadf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:57.315968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a57641-029f-41be-b3fa-143d3a5fadf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:57.316293 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a57641-029f-41be-b3fa-143d3a5fadf5 admin admin] PATH_INFO: `/v3/projects/624bfdf95f98436b908e1a13a228d03e` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:57.318686 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a57641-029f-41be-b3fa-143d3a5fadf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=624bfdf95f98436b908e1a13a228d03e)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:57.320670 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a57641-029f-41be-b3fa-143d3a5fadf5 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:57.380686 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:57.404845 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-c6a57641-029f-41be-b3fa-143d3a5fadf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '624bfdf95f98436b908e1a13a228d03e', 'request_id': 'req-c6a57641-029f-41be-b3fa-143d3a5fadf5'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:57.415981 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-c6a57641-029f-41be-b3fa-143d3a5fadf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '624bfdf95f98436b908e1a13a228d03e', 'request_id': 'req-c6a57641-029f-41be-b3fa-143d3a5fadf5'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:57.427806 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-c6a57641-029f-41be-b3fa-143d3a5fadf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '624bfdf95f98436b908e1a13a228d03e', 'request_id': 'req-c6a57641-029f-41be-b3fa-143d3a5fadf5'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:57.441273 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-c6a57641-029f-41be-b3fa-143d3a5fadf5 admin admin] The token cache is being invalidate because project 624bfdf95f98436b908e1a13a228d03e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:39:57.443668 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-c6a57641-029f-41be-b3fa-143d3a5fadf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:57.444874 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-c6a57641-029f-41be-b3fa-143d3a5fadf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:57.448372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1218/2443] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:39:57 2022] DELETE /identity/v3/projects/624bfdf95f98436b908e1a13a228d03e => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:57.452866 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78f66f-9549-47b1-9f92-36fae94ab933 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:57.453369 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78f66f-9549-47b1-9f92-36fae94ab933 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:57.453473 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78f66f-9549-47b1-9f92-36fae94ab933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:57.474555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-6d78f66f-9549-47b1-9f92-36fae94ab933 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:57.475770 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-6d78f66f-9549-47b1-9f92-36fae94ab933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:57.585723 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1226/2444] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:39:57.594553 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde316c7-a723-4464-8c14-729773bce857 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:57.594913 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde316c7-a723-4464-8c14-729773bce857 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:57.595375 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde316c7-a723-4464-8c14-729773bce857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:57.640734 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-fde316c7-a723-4464-8c14-729773bce857 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:57.644581 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-fde316c7-a723-4464-8c14-729773bce857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:57.662567 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1219/2445] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:39:57.666352 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78f66f-9549-47b1-9f92-36fae94ab933 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:57.679743 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78f66f-9549-47b1-9f92-36fae94ab933 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:57.687466 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f277df7-87d6-4cf6-9fd6-6c628955bead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:57.688326 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f277df7-87d6-4cf6-9fd6-6c628955bead admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:57.688684 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f277df7-87d6-4cf6-9fd6-6c628955bead admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:57.689038 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f277df7-87d6-4cf6-9fd6-6c628955bead admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:57.689847 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f277df7-87d6-4cf6-9fd6-6c628955bead admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:39:57.690232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f277df7-87d6-4cf6-9fd6-6c628955bead admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:57.692567 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f277df7-87d6-4cf6-9fd6-6c628955bead admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:57.721533 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde316c7-a723-4464-8c14-729773bce857 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:57.741311 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1227/2446] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:39:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:39:57.741699 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde316c7-a723-4464-8c14-729773bce857 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:57.750367 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98009393-fa41-49a9-9734-dd8170b05e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:57.751027 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98009393-fa41-49a9-9734-dd8170b05e08 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:57.751317 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98009393-fa41-49a9-9734-dd8170b05e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:57.752028 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98009393-fa41-49a9-9734-dd8170b05e08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:57.752949 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98009393-fa41-49a9-9734-dd8170b05e08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:57.755743 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98009393-fa41-49a9-9734-dd8170b05e08 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:57.790039 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1220/2447] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:39:57 2022] POST /identity/v3/projects => generated 355 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:39:57.794252 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f277df7-87d6-4cf6-9fd6-6c628955bead admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:57.813968 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f277df7-87d6-4cf6-9fd6-6c628955bead admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:57.946002 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ae633-d90d-4863-bfb8-fdcee1e4b479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:57.947213 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ae633-d90d-4863-bfb8-fdcee1e4b479 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:57.947683 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ae633-d90d-4863-bfb8-fdcee1e4b479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:57.948141 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ae633-d90d-4863-bfb8-fdcee1e4b479 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:57.949187 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ae633-d90d-4863-bfb8-fdcee1e4b479 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:57.951289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ae633-d90d-4863-bfb8-fdcee1e4b479 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:57.985818 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1228/2448] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:39:57 2022] POST /identity/v3/users => generated 278 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:39:57.987456 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98009393-fa41-49a9-9734-dd8170b05e08 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:58.001783 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98009393-fa41-49a9-9734-dd8170b05e08 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:58.008802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08daa92c-92bf-4a31-9612-b8447815e7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:58.009851 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08daa92c-92bf-4a31-9612-b8447815e7b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:58.010587 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08daa92c-92bf-4a31-9612-b8447815e7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:58.011056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08daa92c-92bf-4a31-9612-b8447815e7b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:58.012724 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08daa92c-92bf-4a31-9612-b8447815e7b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:58.015243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08daa92c-92bf-4a31-9612-b8447815e7b1 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:58.037492 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ae633-d90d-4863-bfb8-fdcee1e4b479 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:58.048188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1221/2449] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:39:58.050820 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ae633-d90d-4863-bfb8-fdcee1e4b479 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:58.063045 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da01cdb-ae3e-4ccd-904e-3e834f14ea8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:58.063732 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da01cdb-ae3e-4ccd-904e-3e834f14ea8a admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:58.064120 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da01cdb-ae3e-4ccd-904e-3e834f14ea8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:58.064416 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da01cdb-ae3e-4ccd-904e-3e834f14ea8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:58.065457 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da01cdb-ae3e-4ccd-904e-3e834f14ea8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:58.067731 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da01cdb-ae3e-4ccd-904e-3e834f14ea8a admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:58.084917 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1229/2450] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:39:58 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:39:58.085774 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08daa92c-92bf-4a31-9612-b8447815e7b1 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:58.100161 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08daa92c-92bf-4a31-9612-b8447815e7b1 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:58.106429 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db528ef9-891d-419c-b19b-8b187e487ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:58.107432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db528ef9-891d-419c-b19b-8b187e487ef2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:58.107725 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db528ef9-891d-419c-b19b-8b187e487ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:58.108000 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db528ef9-891d-419c-b19b-8b187e487ef2 admin admin] PATH_INFO: `/v3/projects/347d9d75c5064f9fa5b7c6528d464688/users/30b7bb34b99c4cb48e2096808d39ce7a/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:58.135430 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db528ef9-891d-419c-b19b-8b187e487ef2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=347d9d75c5064f9fa5b7c6528d464688, user_id=30b7bb34b99c4cb48e2096808d39ce7a, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:58.139344 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db528ef9-891d-419c-b19b-8b187e487ef2 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:58.168859 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1222/2451] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:39:58 2022] PUT /identity/v3/projects/347d9d75c5064f9fa5b7c6528d464688/users/30b7bb34b99c4cb48e2096808d39ce7a/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:58.178406 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfbfa3a-bf85-4517-954d-d4aa357258f1 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:58.178698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfbfa3a-bf85-4517-954d-d4aa357258f1 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:58.178999 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfbfa3a-bf85-4517-954d-d4aa357258f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:58.218904 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-edfbfa3a-bf85-4517-954d-d4aa357258f1 None None] MFA Rules not processed for user `30b7bb34b99c4cb48e2096808d39ce7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:58.220720 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-edfbfa3a-bf85-4517-954d-d4aa357258f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:58.257209 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:58.366703 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1230/2452] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:58 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:39:58.367208 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db528ef9-891d-419c-b19b-8b187e487ef2 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:58.482192 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db528ef9-891d-419c-b19b-8b187e487ef2 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:58.490535 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d48de-4e59-49b9-af86-26b7795628bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:58.491565 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d48de-4e59-49b9-af86-26b7795628bc admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:58.491987 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d48de-4e59-49b9-af86-26b7795628bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:58.492418 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5d48de-4e59-49b9-af86-26b7795628bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:58.493319 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d48de-4e59-49b9-af86-26b7795628bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:58.495930 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5d48de-4e59-49b9-af86-26b7795628bc admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:58.544203 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1223/2453] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:39:58 2022] POST /identity/v3/projects => generated 357 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:39:58.545655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfbfa3a-bf85-4517-954d-d4aa357258f1 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:58.565879 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfbfa3a-bf85-4517-954d-d4aa357258f1 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:58.650057 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969e5c2-5fd9-49b5-9e7a-c1414331c194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:58.651039 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969e5c2-5fd9-49b5-9e7a-c1414331c194 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:58.651488 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969e5c2-5fd9-49b5-9e7a-c1414331c194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:58.651766 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969e5c2-5fd9-49b5-9e7a-c1414331c194 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:58.652520 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969e5c2-5fd9-49b5-9e7a-c1414331c194 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:58.654489 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969e5c2-5fd9-49b5-9e7a-c1414331c194 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:58.682849 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1231/2454] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:39:58 2022] POST /identity/v3/users => generated 285 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:39:58.684125 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d48de-4e59-49b9-af86-26b7795628bc admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:58.697508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5d48de-4e59-49b9-af86-26b7795628bc admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:58.706555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff47ae-4df9-40d0-8489-ee41b890dc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:58.707874 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff47ae-4df9-40d0-8489-ee41b890dc70 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:58.708860 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff47ae-4df9-40d0-8489-ee41b890dc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:58.709156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff47ae-4df9-40d0-8489-ee41b890dc70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:58.709900 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff47ae-4df9-40d0-8489-ee41b890dc70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:58.712087 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff47ae-4df9-40d0-8489-ee41b890dc70 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:58.726599 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1224/2455] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:39:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:39:58.730509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969e5c2-5fd9-49b5-9e7a-c1414331c194 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:58.742411 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969e5c2-5fd9-49b5-9e7a-c1414331c194 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:58.753672 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67378175-44fb-4214-96df-fbce7cc22e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:58.755324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67378175-44fb-4214-96df-fbce7cc22e07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:58.755755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67378175-44fb-4214-96df-fbce7cc22e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:58.756149 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67378175-44fb-4214-96df-fbce7cc22e07 admin admin] PATH_INFO: `/v3/projects/9124993073634c6eaa41a90c0710f12a/users/3e71c079db234c1db9366f57121c62b4/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:58.783552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67378175-44fb-4214-96df-fbce7cc22e07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9124993073634c6eaa41a90c0710f12a, user_id=3e71c079db234c1db9366f57121c62b4, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:58.787624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67378175-44fb-4214-96df-fbce7cc22e07 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:58.809350 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1232/2456] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:39:58 2022] PUT /identity/v3/projects/9124993073634c6eaa41a90c0710f12a/users/3e71c079db234c1db9366f57121c62b4/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:58.810554 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff47ae-4df9-40d0-8489-ee41b890dc70 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:58.824283 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff47ae-4df9-40d0-8489-ee41b890dc70 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:58.867627 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:58.913642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65bcf7f-d534-429e-aa25-16ec19743531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:58.914362 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bcf7f-d534-429e-aa25-16ec19743531 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:58.914760 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bcf7f-d534-429e-aa25-16ec19743531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:58.915154 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bcf7f-d534-429e-aa25-16ec19743531 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:58.916018 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65bcf7f-d534-429e-aa25-16ec19743531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:58.917831 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65bcf7f-d534-429e-aa25-16ec19743531 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:58.933882 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1225/2457] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:39:58 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:39:58.937994 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67378175-44fb-4214-96df-fbce7cc22e07 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:58.954870 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67378175-44fb-4214-96df-fbce7cc22e07 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:58.961391 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe62a38e-a6f3-4ced-b13e-819c23e75f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:58.962798 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62a38e-a6f3-4ced-b13e-819c23e75f04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:58.963322 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62a38e-a6f3-4ced-b13e-819c23e75f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:58.963979 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62a38e-a6f3-4ced-b13e-819c23e75f04 admin admin] PATH_INFO: `/v3/projects/9124993073634c6eaa41a90c0710f12a/users/3e71c079db234c1db9366f57121c62b4/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:58.966208 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe62a38e-a6f3-4ced-b13e-819c23e75f04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9124993073634c6eaa41a90c0710f12a, user_id=3e71c079db234c1db9366f57121c62b4, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:58.969329 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe62a38e-a6f3-4ced-b13e-819c23e75f04 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:59.010788 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ed659-d088-4654-ae13-58ef1ed76f6b None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:59.010788 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ed659-d088-4654-ae13-58ef1ed76f6b None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:59.010788 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ed659-d088-4654-ae13-58ef1ed76f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:59.013277 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1233/2458] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:39:58 2022] PUT /identity/v3/projects/9124993073634c6eaa41a90c0710f12a/users/3e71c079db234c1db9366f57121c62b4/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:59.027019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe62a38e-a6f3-4ced-b13e-819c23e75f04 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:59.037109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-fe62a38e-a6f3-4ced-b13e-819c23e75f04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:59.064363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-443ed659-d088-4654-ae13-58ef1ed76f6b None None] MFA Rules not processed for user `3e71c079db234c1db9366f57121c62b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:59.065812 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-443ed659-d088-4654-ae13-58ef1ed76f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:59.149247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe62a38e-a6f3-4ced-b13e-819c23e75f04 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:59.154036 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51216ec3-75a8-463f-9015-84e642098ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:59.161200 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51216ec3-75a8-463f-9015-84e642098ce2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:59.162022 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51216ec3-75a8-463f-9015-84e642098ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:59.162584 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51216ec3-75a8-463f-9015-84e642098ce2 admin admin] PATH_INFO: `/v3/users/7ea2ed0a883248948c47f188c54bd42a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:59.164830 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51216ec3-75a8-463f-9015-84e642098ce2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ea2ed0a883248948c47f188c54bd42a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:59.166794 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51216ec3-75a8-463f-9015-84e642098ce2 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:59.182935 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1226/2459] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:59 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:39:59.194908 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:59.198100 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443ed659-d088-4654-ae13-58ef1ed76f6b None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:59.200329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-443ed659-d088-4654-ae13-58ef1ed76f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:59.238828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-51216ec3-75a8-463f-9015-84e642098ce2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ea2ed0a883248948c47f188c54bd42a'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:59.253431 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-51216ec3-75a8-463f-9015-84e642098ce2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ea2ed0a883248948c47f188c54bd42a'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:59.271188 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-51216ec3-75a8-463f-9015-84e642098ce2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ea2ed0a883248948c47f188c54bd42a'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:59.271956 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-51216ec3-75a8-463f-9015-84e642098ce2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ea2ed0a883248948c47f188c54bd42a'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:59.272650 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-51216ec3-75a8-463f-9015-84e642098ce2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ea2ed0a883248948c47f188c54bd42a'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:39:59.289980 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1234/2460] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:39:59 2022] DELETE /identity/v3/users/7ea2ed0a883248948c47f188c54bd42a => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:39:59.295892 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fcfc3-9d4d-4117-8fa5-8df0ac64d1a4 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:59.296213 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fcfc3-9d4d-4117-8fa5-8df0ac64d1a4 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:59.299918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fcfc3-9d4d-4117-8fa5-8df0ac64d1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:59.300128 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443ed659-d088-4654-ae13-58ef1ed76f6b None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:59.320130 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-7e9fcfc3-9d4d-4117-8fa5-8df0ac64d1a4 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:39:59.321519 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-7e9fcfc3-9d4d-4117-8fa5-8df0ac64d1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:59.381994 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0316e86-21b2-4dec-8975-f9f19507fe5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:59.383153 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0316e86-21b2-4dec-8975-f9f19507fe5e service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:59.383759 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0316e86-21b2-4dec-8975-f9f19507fe5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:59.385200 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0316e86-21b2-4dec-8975-f9f19507fe5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:59.387838 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-c0316e86-21b2-4dec-8975-f9f19507fe5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:59.419547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:59.452983 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1235/2461] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:39:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:39:59.462262 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fcfc3-9d4d-4117-8fa5-8df0ac64d1a4 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:59.464106 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-7e9fcfc3-9d4d-4117-8fa5-8df0ac64d1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:59.554023 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fcfc3-9d4d-4117-8fa5-8df0ac64d1a4 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:59.558852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00257b09-66b8-4073-a55a-2e9257df96e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:59.559636 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00257b09-66b8-4073-a55a-2e9257df96e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:59.559998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00257b09-66b8-4073-a55a-2e9257df96e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:59.584260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00257b09-66b8-4073-a55a-2e9257df96e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:59.584260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00257b09-66b8-4073-a55a-2e9257df96e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:59.584260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00257b09-66b8-4073-a55a-2e9257df96e4 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:59.596642 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1236/2462] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:39:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:39:59.612959 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0316e86-21b2-4dec-8975-f9f19507fe5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:59.615529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0316e86-21b2-4dec-8975-f9f19507fe5e service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:59.644530 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1227/2463] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:39:59 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:39:59.774461 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00257b09-66b8-4073-a55a-2e9257df96e4 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:39:59.776026 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-00257b09-66b8-4073-a55a-2e9257df96e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:59.797762 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00257b09-66b8-4073-a55a-2e9257df96e4 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:39:59.804488 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330181bf-2997-41ef-8f58-f86a6389cf61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:39:59.805576 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330181bf-2997-41ef-8f58-f86a6389cf61 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:39:59.806884 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330181bf-2997-41ef-8f58-f86a6389cf61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:39:59.807375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330181bf-2997-41ef-8f58-f86a6389cf61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:39:59.810930 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-330181bf-2997-41ef-8f58-f86a6389cf61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:39:59.863519 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:39:59.928564 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330181bf-2997-41ef-8f58-f86a6389cf61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:39:59.930712 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330181bf-2997-41ef-8f58-f86a6389cf61 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:39:59.944367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1237/2464] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:39:59 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:40:00.080459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0316e86-21b2-4dec-8975-f9f19507fe5e service cinder] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:00.082800 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-c0316e86-21b2-4dec-8975-f9f19507fe5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:00.132648 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0316e86-21b2-4dec-8975-f9f19507fe5e service cinder] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:00.141381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78e50d-a4bd-4db0-aad1-fe14b15418ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:00.142597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78e50d-a4bd-4db0-aad1-fe14b15418ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:00.142889 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78e50d-a4bd-4db0-aad1-fe14b15418ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:00.143716 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78e50d-a4bd-4db0-aad1-fe14b15418ca admin admin] PATH_INFO: `/v3/projects/3405fd7245fc467ea1938f9251662f0d` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:00.146083 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e78e50d-a4bd-4db0-aad1-fe14b15418ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=3405fd7245fc467ea1938f9251662f0d)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:00.149921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e78e50d-a4bd-4db0-aad1-fe14b15418ca admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:00.220636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2e78e50d-a4bd-4db0-aad1-fe14b15418ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3405fd7245fc467ea1938f9251662f0d', 'request_id': 'req-2e78e50d-a4bd-4db0-aad1-fe14b15418ca'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:00.230156 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2e78e50d-a4bd-4db0-aad1-fe14b15418ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3405fd7245fc467ea1938f9251662f0d', 'request_id': 'req-2e78e50d-a4bd-4db0-aad1-fe14b15418ca'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:00.237977 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2e78e50d-a4bd-4db0-aad1-fe14b15418ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3405fd7245fc467ea1938f9251662f0d', 'request_id': 'req-2e78e50d-a4bd-4db0-aad1-fe14b15418ca'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:00.251764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2e78e50d-a4bd-4db0-aad1-fe14b15418ca admin admin] The token cache is being invalidate because project 3405fd7245fc467ea1938f9251662f0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:40:00.252062 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2e78e50d-a4bd-4db0-aad1-fe14b15418ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:00.254786 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2e78e50d-a4bd-4db0-aad1-fe14b15418ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:00.257285 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1228/2465] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:40:00 2022] DELETE /identity/v3/projects/3405fd7245fc467ea1938f9251662f0d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:00.262066 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d68360-b6d2-4414-b293-908c80122253 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:00.262361 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d68360-b6d2-4414-b293-908c80122253 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:00.262646 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d68360-b6d2-4414-b293-908c80122253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:00.286958 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-27d68360-b6d2-4414-b293-908c80122253 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:00.288556 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-27d68360-b6d2-4414-b293-908c80122253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:00.370701 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1238/2466] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:00.378704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537490e2-5475-4097-8f2b-cca2f4e07308 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:00.379593 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537490e2-5475-4097-8f2b-cca2f4e07308 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:00.379942 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537490e2-5475-4097-8f2b-cca2f4e07308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:00.400564 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-537490e2-5475-4097-8f2b-cca2f4e07308 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:00.401787 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-537490e2-5475-4097-8f2b-cca2f4e07308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:00.412201 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1229/2467] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:00.415548 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d68360-b6d2-4414-b293-908c80122253 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:00.425460 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d68360-b6d2-4414-b293-908c80122253 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:00.431005 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0917ae5-c0fb-46d2-82f7-88839fadd114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:00.431682 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0917ae5-c0fb-46d2-82f7-88839fadd114 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:00.431965 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0917ae5-c0fb-46d2-82f7-88839fadd114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:00.432267 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0917ae5-c0fb-46d2-82f7-88839fadd114 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:00.433000 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0917ae5-c0fb-46d2-82f7-88839fadd114 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:40:00.433314 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0917ae5-c0fb-46d2-82f7-88839fadd114 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:00.434906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0917ae5-c0fb-46d2-82f7-88839fadd114 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:00.448777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1239/2468] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:40:00.453359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537490e2-5475-4097-8f2b-cca2f4e07308 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:00.464438 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537490e2-5475-4097-8f2b-cca2f4e07308 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:00.469490 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5bb05e-4551-4216-b244-9843c8db797a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:00.470223 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5bb05e-4551-4216-b244-9843c8db797a admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:00.470720 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5bb05e-4551-4216-b244-9843c8db797a admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:00.471009 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5bb05e-4551-4216-b244-9843c8db797a admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:00.471867 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5bb05e-4551-4216-b244-9843c8db797a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:00.474419 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5bb05e-4551-4216-b244-9843c8db797a admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:00.504524 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1230/2469] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:40:00 2022] POST /identity/v3/projects => generated 405 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:40:00.508561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0917ae5-c0fb-46d2-82f7-88839fadd114 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:00.520526 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0917ae5-c0fb-46d2-82f7-88839fadd114 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:00.597516 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865e96e-8942-4fe3-a9f8-fdbc79792c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:00.598913 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865e96e-8942-4fe3-a9f8-fdbc79792c59 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:00.599196 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865e96e-8942-4fe3-a9f8-fdbc79792c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:00.599474 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865e96e-8942-4fe3-a9f8-fdbc79792c59 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:00.600189 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8865e96e-8942-4fe3-a9f8-fdbc79792c59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:00.602165 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8865e96e-8942-4fe3-a9f8-fdbc79792c59 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:00.630880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1240/2470] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:40:00 2022] POST /identity/v3/users => generated 303 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:40:00.632125 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5bb05e-4551-4216-b244-9843c8db797a admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:00.644198 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5bb05e-4551-4216-b244-9843c8db797a admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:00.650215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7975e08-b349-4d3b-8fbd-827b841fc107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:00.650967 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7975e08-b349-4d3b-8fbd-827b841fc107 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:00.651457 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7975e08-b349-4d3b-8fbd-827b841fc107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:00.651814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7975e08-b349-4d3b-8fbd-827b841fc107 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:00.652725 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7975e08-b349-4d3b-8fbd-827b841fc107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:00.654499 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7975e08-b349-4d3b-8fbd-827b841fc107 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:00.666737 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1231/2471] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:40:00.671236 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865e96e-8942-4fe3-a9f8-fdbc79792c59 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:00.681962 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865e96e-8942-4fe3-a9f8-fdbc79792c59 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:00.686901 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac47b7e-a0cd-4560-b8b5-8a5a03b9f59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:00.687641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac47b7e-a0cd-4560-b8b5-8a5a03b9f59a admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:00.687950 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac47b7e-a0cd-4560-b8b5-8a5a03b9f59a admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:00.688468 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac47b7e-a0cd-4560-b8b5-8a5a03b9f59a admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:00.689275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac47b7e-a0cd-4560-b8b5-8a5a03b9f59a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:00.690814 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac47b7e-a0cd-4560-b8b5-8a5a03b9f59a admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:00.702644 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1241/2472] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:40:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:40:00.706884 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7975e08-b349-4d3b-8fbd-827b841fc107 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:00.720694 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7975e08-b349-4d3b-8fbd-827b841fc107 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:00.726900 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5f3b8-be0f-4400-9a24-df88d5b5d373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:00.727770 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5f3b8-be0f-4400-9a24-df88d5b5d373 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:00.728064 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5f3b8-be0f-4400-9a24-df88d5b5d373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:00.728486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5f3b8-be0f-4400-9a24-df88d5b5d373 admin admin] PATH_INFO: `/v3/projects/ebc26cab9e0747fe94e4fdd95d99088d/users/8b92b69e7e0a44908adf4002232dc380/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:00.747305 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf5f3b8-be0f-4400-9a24-df88d5b5d373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebc26cab9e0747fe94e4fdd95d99088d, user_id=8b92b69e7e0a44908adf4002232dc380, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:00.750266 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf5f3b8-be0f-4400-9a24-df88d5b5d373 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:00.769209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1232/2473] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:40:00 2022] PUT /identity/v3/projects/ebc26cab9e0747fe94e4fdd95d99088d/users/8b92b69e7e0a44908adf4002232dc380/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:00.776891 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c1a80-4831-4a44-90e3-16affc7e494a None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:00.777196 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c1a80-4831-4a44-90e3-16affc7e494a None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:00.777551 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c1a80-4831-4a44-90e3-16affc7e494a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:00.799550 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-d00c1a80-4831-4a44-90e3-16affc7e494a None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:00.801351 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d00c1a80-4831-4a44-90e3-16affc7e494a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:00.871696 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:00.930484 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1242/2474] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:00.943457 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5f3b8-be0f-4400-9a24-df88d5b5d373 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:00.945388 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-daf5f3b8-be0f-4400-9a24-df88d5b5d373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:01.031997 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:01.093199 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5f3b8-be0f-4400-9a24-df88d5b5d373 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:01.099874 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe7213-d60b-4606-b75c-6d7290964e00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:01.100866 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe7213-d60b-4606-b75c-6d7290964e00 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:01.101072 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe7213-d60b-4606-b75c-6d7290964e00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:01.101357 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe7213-d60b-4606-b75c-6d7290964e00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:01.113035 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fe7213-d60b-4606-b75c-6d7290964e00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:01.115226 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fe7213-d60b-4606-b75c-6d7290964e00 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:01.131437 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1233/2475] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:40:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:40:02.067094 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c6e1a-7807-4367-9f7b-82e248819c47 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:02.067996 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c6e1a-7807-4367-9f7b-82e248819c47 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:02.068469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c6e1a-7807-4367-9f7b-82e248819c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:02.123031 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-089c6e1a-7807-4367-9f7b-82e248819c47 None None] MFA Rules not processed for user `8b92b69e7e0a44908adf4002232dc380`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:02.124482 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-089c6e1a-7807-4367-9f7b-82e248819c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:02.277864 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1243/2476] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:02 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:02.282641 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe7213-d60b-4606-b75c-6d7290964e00 service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:02.297760 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe7213-d60b-4606-b75c-6d7290964e00 service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:02.309608 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889eea18-17ae-4458-8030-6fbdda17d6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:02.310258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889eea18-17ae-4458-8030-6fbdda17d6e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:02.310552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889eea18-17ae-4458-8030-6fbdda17d6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:02.310841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889eea18-17ae-4458-8030-6fbdda17d6e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:02.311807 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889eea18-17ae-4458-8030-6fbdda17d6e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:02.313835 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889eea18-17ae-4458-8030-6fbdda17d6e4 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:02.368559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1234/2477] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:40:02 2022] POST /identity/v3/projects => generated 403 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:40:02.370563 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c6e1a-7807-4367-9f7b-82e248819c47 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:02.392586 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c6e1a-7807-4367-9f7b-82e248819c47 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:02.501040 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f107a-25d8-4b78-89af-98c2f2bfcef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:02.501850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f107a-25d8-4b78-89af-98c2f2bfcef9 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:02.502246 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f107a-25d8-4b78-89af-98c2f2bfcef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:02.503490 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f107a-25d8-4b78-89af-98c2f2bfcef9 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:02.504631 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f107a-25d8-4b78-89af-98c2f2bfcef9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:02.507932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f107a-25d8-4b78-89af-98c2f2bfcef9 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:02.563678 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1244/2478] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:40:02 2022] POST /identity/v3/users => generated 308 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:40:02.567718 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889eea18-17ae-4458-8030-6fbdda17d6e4 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:02.586735 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889eea18-17ae-4458-8030-6fbdda17d6e4 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:02.601093 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aeccc2-6354-4ba5-b9bf-0270e7f6a164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:02.601992 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aeccc2-6354-4ba5-b9bf-0270e7f6a164 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:02.603852 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aeccc2-6354-4ba5-b9bf-0270e7f6a164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:02.604346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aeccc2-6354-4ba5-b9bf-0270e7f6a164 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:02.605450 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aeccc2-6354-4ba5-b9bf-0270e7f6a164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:02.610356 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aeccc2-6354-4ba5-b9bf-0270e7f6a164 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:02.635275 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1235/2479] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:40:02.640136 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f107a-25d8-4b78-89af-98c2f2bfcef9 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:02.654433 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f107a-25d8-4b78-89af-98c2f2bfcef9 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:02.662531 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6262c-e7ef-4c52-a195-e009d32a980e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:02.663579 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6262c-e7ef-4c52-a195-e009d32a980e admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:02.664015 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6262c-e7ef-4c52-a195-e009d32a980e admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:02.664395 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6262c-e7ef-4c52-a195-e009d32a980e admin admin] PATH_INFO: `/v3/projects/203a9d7ee0f34bcda24d33e251b5f6f1/users/0242005dd5bc4430baf5eba9be572751/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:02.684713 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6262c-e7ef-4c52-a195-e009d32a980e admin admin] RBAC: Authorizing `identity:create_grant(project_id=203a9d7ee0f34bcda24d33e251b5f6f1, user_id=0242005dd5bc4430baf5eba9be572751, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:02.687445 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6262c-e7ef-4c52-a195-e009d32a980e admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:02.712106 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1245/2480] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:40:02 2022] PUT /identity/v3/projects/203a9d7ee0f34bcda24d33e251b5f6f1/users/0242005dd5bc4430baf5eba9be572751/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:40:02.716097 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aeccc2-6354-4ba5-b9bf-0270e7f6a164 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:02.731998 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aeccc2-6354-4ba5-b9bf-0270e7f6a164 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:02.849321 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7569ac-295a-4ec1-9594-137d30b8107b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:02.850109 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7569ac-295a-4ec1-9594-137d30b8107b admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:02.850426 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7569ac-295a-4ec1-9594-137d30b8107b admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:02.851078 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7569ac-295a-4ec1-9594-137d30b8107b admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:02.851825 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7569ac-295a-4ec1-9594-137d30b8107b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:02.853753 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7569ac-295a-4ec1-9594-137d30b8107b admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:02.867381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1236/2481] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:40:02.873898 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6262c-e7ef-4c52-a195-e009d32a980e admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:02.888601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6262c-e7ef-4c52-a195-e009d32a980e admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:02.905116 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495ad849-08f7-4d07-bc1d-6c3a3a34c053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:02.906861 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ad849-08f7-4d07-bc1d-6c3a3a34c053 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:02.907839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ad849-08f7-4d07-bc1d-6c3a3a34c053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:02.908395 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ad849-08f7-4d07-bc1d-6c3a3a34c053 admin admin] PATH_INFO: `/v3/projects/203a9d7ee0f34bcda24d33e251b5f6f1/users/0242005dd5bc4430baf5eba9be572751/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:02.912296 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495ad849-08f7-4d07-bc1d-6c3a3a34c053 admin admin] RBAC: Authorizing `identity:create_grant(project_id=203a9d7ee0f34bcda24d33e251b5f6f1, user_id=0242005dd5bc4430baf5eba9be572751, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:02.915548 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495ad849-08f7-4d07-bc1d-6c3a3a34c053 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:02.929984 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1246/2482] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:40:02 2022] PUT /identity/v3/projects/203a9d7ee0f34bcda24d33e251b5f6f1/users/0242005dd5bc4430baf5eba9be572751/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:03.911752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416efeea-2d0f-419a-acf9-a0d13a466463 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:03.912276 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416efeea-2d0f-419a-acf9-a0d13a466463 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:03.912430 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416efeea-2d0f-419a-acf9-a0d13a466463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:03.963281 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-416efeea-2d0f-419a-acf9-a0d13a466463 None None] MFA Rules not processed for user `0242005dd5bc4430baf5eba9be572751`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:03.966046 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-416efeea-2d0f-419a-acf9-a0d13a466463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:04.116319 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1237/2483] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:03 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:04.734773 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495ad849-08f7-4d07-bc1d-6c3a3a34c053 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:04.737130 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-495ad849-08f7-4d07-bc1d-6c3a3a34c053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:04.804678 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:04.935759 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495ad849-08f7-4d07-bc1d-6c3a3a34c053 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:04.945882 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64597a-a6ff-4d4c-81ee-ff0826bc4285 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc0f137e0d24dd2a460689bc17a1b98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:04.947018 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64597a-a6ff-4d4c-81ee-ff0826bc4285 service placement] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:04.947984 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64597a-a6ff-4d4c-81ee-ff0826bc4285 service placement] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:04.948839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64597a-a6ff-4d4c-81ee-ff0826bc4285 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:04.958469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-2e64597a-a6ff-4d4c-81ee-ff0826bc4285 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:04.983024 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64597a-a6ff-4d4c-81ee-ff0826bc4285 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:04.986175 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64597a-a6ff-4d4c-81ee-ff0826bc4285 service placement] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:05.003351 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1247/2484] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:40:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:40:05.771501 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416efeea-2d0f-419a-acf9-a0d13a466463 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:05.807052 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:05.843853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416efeea-2d0f-419a-acf9-a0d13a466463 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:05.859464 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df32a233-121f-45b9-be89-c5800a924056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:05.859464 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32a233-121f-45b9-be89-c5800a924056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:05.859464 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32a233-121f-45b9-be89-c5800a924056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:05.859464 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32a233-121f-45b9-be89-c5800a924056 admin admin] PATH_INFO: `/v3/users/8b92b69e7e0a44908adf4002232dc380` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:05.860321 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df32a233-121f-45b9-be89-c5800a924056 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b92b69e7e0a44908adf4002232dc380)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:05.861665 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df32a233-121f-45b9-be89-c5800a924056 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:05.922205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-df32a233-121f-45b9-be89-c5800a924056 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b92b69e7e0a44908adf4002232dc380'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:05.922878 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-df32a233-121f-45b9-be89-c5800a924056 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b92b69e7e0a44908adf4002232dc380'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:05.930822 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-df32a233-121f-45b9-be89-c5800a924056 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b92b69e7e0a44908adf4002232dc380'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:05.948661 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-df32a233-121f-45b9-be89-c5800a924056 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b92b69e7e0a44908adf4002232dc380'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:05.958375 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-df32a233-121f-45b9-be89-c5800a924056 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b92b69e7e0a44908adf4002232dc380'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:05.968876 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1238/2485] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:40:05 2022] DELETE /identity/v3/users/8b92b69e7e0a44908adf4002232dc380 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:40:06.234180 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64597a-a6ff-4d4c-81ee-ff0826bc4285 service placement] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:06.258164 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64597a-a6ff-4d4c-81ee-ff0826bc4285 service placement] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:06.321427 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb307dc-7dd7-4a22-adc8-2dcd9c7afeed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:06.322298 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb307dc-7dd7-4a22-adc8-2dcd9c7afeed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:06.322582 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb307dc-7dd7-4a22-adc8-2dcd9c7afeed admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:06.322861 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb307dc-7dd7-4a22-adc8-2dcd9c7afeed admin admin] PATH_INFO: `/v3/projects/ebc26cab9e0747fe94e4fdd95d99088d` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:06.324020 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb307dc-7dd7-4a22-adc8-2dcd9c7afeed admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebc26cab9e0747fe94e4fdd95d99088d)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:06.326245 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb307dc-7dd7-4a22-adc8-2dcd9c7afeed admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:06.375253 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-fbb307dc-7dd7-4a22-adc8-2dcd9c7afeed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebc26cab9e0747fe94e4fdd95d99088d', 'request_id': 'req-fbb307dc-7dd7-4a22-adc8-2dcd9c7afeed'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:06.381729 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-fbb307dc-7dd7-4a22-adc8-2dcd9c7afeed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebc26cab9e0747fe94e4fdd95d99088d', 'request_id': 'req-fbb307dc-7dd7-4a22-adc8-2dcd9c7afeed'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:06.388186 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-fbb307dc-7dd7-4a22-adc8-2dcd9c7afeed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebc26cab9e0747fe94e4fdd95d99088d', 'request_id': 'req-fbb307dc-7dd7-4a22-adc8-2dcd9c7afeed'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:06.403395 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-fbb307dc-7dd7-4a22-adc8-2dcd9c7afeed admin admin] The token cache is being invalidate because project ebc26cab9e0747fe94e4fdd95d99088d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:40:06.403683 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-fbb307dc-7dd7-4a22-adc8-2dcd9c7afeed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:06.404352 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-fbb307dc-7dd7-4a22-adc8-2dcd9c7afeed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:06.405905 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1248/2486] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:40:06 2022] DELETE /identity/v3/projects/ebc26cab9e0747fe94e4fdd95d99088d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:06.410476 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df32a233-121f-45b9-be89-c5800a924056 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:06.411924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-df32a233-121f-45b9-be89-c5800a924056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:06.426702 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df32a233-121f-45b9-be89-c5800a924056 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:06.486809 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce26b9f-9966-423b-ae8b-45c4d4744271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:06.487715 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce26b9f-9966-423b-ae8b-45c4d4744271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:06.488565 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce26b9f-9966-423b-ae8b-45c4d4744271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:06.488937 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce26b9f-9966-423b-ae8b-45c4d4744271 admin admin] PATH_INFO: `/v3/users/0242005dd5bc4430baf5eba9be572751` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:06.490265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce26b9f-9966-423b-ae8b-45c4d4744271 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0242005dd5bc4430baf5eba9be572751)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:06.492287 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce26b9f-9966-423b-ae8b-45c4d4744271 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:06.544547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bce26b9f-9966-423b-ae8b-45c4d4744271 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0242005dd5bc4430baf5eba9be572751'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:06.545263 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bce26b9f-9966-423b-ae8b-45c4d4744271 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0242005dd5bc4430baf5eba9be572751'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:06.547451 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:06.555791 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bce26b9f-9966-423b-ae8b-45c4d4744271 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0242005dd5bc4430baf5eba9be572751'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:06.569790 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bce26b9f-9966-423b-ae8b-45c4d4744271 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0242005dd5bc4430baf5eba9be572751'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:06.580508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bce26b9f-9966-423b-ae8b-45c4d4744271 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0242005dd5bc4430baf5eba9be572751'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:06.587341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1239/2487] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:40:06 2022] DELETE /identity/v3/users/0242005dd5bc4430baf5eba9be572751 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:40:06.788048 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb307dc-7dd7-4a22-adc8-2dcd9c7afeed admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:06.789746 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-fbb307dc-7dd7-4a22-adc8-2dcd9c7afeed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:06.818192 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb307dc-7dd7-4a22-adc8-2dcd9c7afeed admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:06.866403 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:06.884833 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9352a-f573-466c-a024-e7eabbf5d1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:06.885798 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9352a-f573-466c-a024-e7eabbf5d1d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:06.886048 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9352a-f573-466c-a024-e7eabbf5d1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:06.886319 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9352a-f573-466c-a024-e7eabbf5d1d5 admin admin] PATH_INFO: `/v3/projects/203a9d7ee0f34bcda24d33e251b5f6f1` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:06.887583 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9352a-f573-466c-a024-e7eabbf5d1d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=203a9d7ee0f34bcda24d33e251b5f6f1)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:06.889570 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9352a-f573-466c-a024-e7eabbf5d1d5 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:06.943620 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ebc9352a-f573-466c-a024-e7eabbf5d1d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '203a9d7ee0f34bcda24d33e251b5f6f1', 'request_id': 'req-ebc9352a-f573-466c-a024-e7eabbf5d1d5'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:06.950827 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ebc9352a-f573-466c-a024-e7eabbf5d1d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '203a9d7ee0f34bcda24d33e251b5f6f1', 'request_id': 'req-ebc9352a-f573-466c-a024-e7eabbf5d1d5'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:06.960574 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ebc9352a-f573-466c-a024-e7eabbf5d1d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '203a9d7ee0f34bcda24d33e251b5f6f1', 'request_id': 'req-ebc9352a-f573-466c-a024-e7eabbf5d1d5'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:06.979499 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ebc9352a-f573-466c-a024-e7eabbf5d1d5 admin admin] The token cache is being invalidate because project 203a9d7ee0f34bcda24d33e251b5f6f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:40:06.979958 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ebc9352a-f573-466c-a024-e7eabbf5d1d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:06.980917 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-ebc9352a-f573-466c-a024-e7eabbf5d1d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:06.983132 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1249/2488] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:40:06 2022] DELETE /identity/v3/projects/203a9d7ee0f34bcda24d33e251b5f6f1 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:06.990999 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e0797-6ee7-45bc-bd44-52a4f625b211 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:06.991442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e0797-6ee7-45bc-bd44-52a4f625b211 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:06.992067 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e0797-6ee7-45bc-bd44-52a4f625b211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:07.019384 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-f35e0797-6ee7-45bc-bd44-52a4f625b211 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:07.021344 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-f35e0797-6ee7-45bc-bd44-52a4f625b211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:07.126980 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1240/2489] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:07.132869 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4af1c-1eae-4e6b-bd12-03db2443993e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:07.133160 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4af1c-1eae-4e6b-bd12-03db2443993e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:07.133477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4af1c-1eae-4e6b-bd12-03db2443993e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:07.153745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-8fd4af1c-1eae-4e6b-bd12-03db2443993e None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:07.155110 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-8fd4af1c-1eae-4e6b-bd12-03db2443993e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:07.164457 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1250/2490] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:07.169187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e0797-6ee7-45bc-bd44-52a4f625b211 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:07.179248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e0797-6ee7-45bc-bd44-52a4f625b211 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:07.184564 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89813ddb-633c-4f54-8d10-cc9889ab0ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:07.185793 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89813ddb-633c-4f54-8d10-cc9889ab0ecc admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:07.185793 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89813ddb-633c-4f54-8d10-cc9889ab0ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:07.185793 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89813ddb-633c-4f54-8d10-cc9889ab0ecc admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:07.186509 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89813ddb-633c-4f54-8d10-cc9889ab0ecc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:40:07.186823 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89813ddb-633c-4f54-8d10-cc9889ab0ecc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:07.188469 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89813ddb-633c-4f54-8d10-cc9889ab0ecc admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:07.207703 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1241/2491] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:40:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:40:07.214383 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4af1c-1eae-4e6b-bd12-03db2443993e None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:07.230499 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4af1c-1eae-4e6b-bd12-03db2443993e None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:07.246652 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4f2fd-7fb8-4648-8be0-10309b737ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:07.247607 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4f2fd-7fb8-4648-8be0-10309b737ce2 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:07.247770 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4f2fd-7fb8-4648-8be0-10309b737ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:07.248075 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4f2fd-7fb8-4648-8be0-10309b737ce2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:07.249126 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf4f2fd-7fb8-4648-8be0-10309b737ce2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:07.251343 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf4f2fd-7fb8-4648-8be0-10309b737ce2 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:07.292225 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1251/2492] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:40:07 2022] POST /identity/v3/projects => generated 369 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:40:07.295046 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89813ddb-633c-4f54-8d10-cc9889ab0ecc admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:07.323391 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89813ddb-633c-4f54-8d10-cc9889ab0ecc admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:07.451641 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739eb1d3-8979-4d17-8e17-86c49aa7a672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:07.453057 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739eb1d3-8979-4d17-8e17-86c49aa7a672 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:07.453395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739eb1d3-8979-4d17-8e17-86c49aa7a672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:07.454247 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739eb1d3-8979-4d17-8e17-86c49aa7a672 admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:07.455584 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739eb1d3-8979-4d17-8e17-86c49aa7a672 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:07.460877 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739eb1d3-8979-4d17-8e17-86c49aa7a672 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:07.492212 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1242/2493] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:40:07 2022] POST /identity/v3/users => generated 285 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:40:07.498220 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4f2fd-7fb8-4648-8be0-10309b737ce2 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:07.528754 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4f2fd-7fb8-4648-8be0-10309b737ce2 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:07.543037 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b04aed-4b25-42f4-a7b5-1724c6062fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:07.543880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b04aed-4b25-42f4-a7b5-1724c6062fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:07.544392 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b04aed-4b25-42f4-a7b5-1724c6062fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:07.545262 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b04aed-4b25-42f4-a7b5-1724c6062fd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:07.548926 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b04aed-4b25-42f4-a7b5-1724c6062fd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:07.548926 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b04aed-4b25-42f4-a7b5-1724c6062fd9 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:07.574012 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1252/2494] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:40:07.579800 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739eb1d3-8979-4d17-8e17-86c49aa7a672 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:07.599162 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739eb1d3-8979-4d17-8e17-86c49aa7a672 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:07.611884 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5290bf79-1084-4833-ac8e-aaf5940609e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:07.611884 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290bf79-1084-4833-ac8e-aaf5940609e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:07.612432 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290bf79-1084-4833-ac8e-aaf5940609e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:07.612776 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290bf79-1084-4833-ac8e-aaf5940609e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:07.614311 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5290bf79-1084-4833-ac8e-aaf5940609e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:07.617969 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5290bf79-1084-4833-ac8e-aaf5940609e9 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:07.664401 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1243/2495] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:40:07.667086 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b04aed-4b25-42f4-a7b5-1724c6062fd9 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:07.686745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b04aed-4b25-42f4-a7b5-1724c6062fd9 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:07.693336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0917d8-7bb3-4329-ba56-21f0c187046f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:07.695256 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0917d8-7bb3-4329-ba56-21f0c187046f admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:07.695543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0917d8-7bb3-4329-ba56-21f0c187046f admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:07.697721 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0917d8-7bb3-4329-ba56-21f0c187046f admin admin] PATH_INFO: `/v3/projects/d588b49ea77c4393a6cadf4b0679d8e6/users/cd10a1f4111d4291915d25ad08c1afb9/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:07.716578 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0917d8-7bb3-4329-ba56-21f0c187046f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d588b49ea77c4393a6cadf4b0679d8e6, user_id=cd10a1f4111d4291915d25ad08c1afb9, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:07.719473 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0917d8-7bb3-4329-ba56-21f0c187046f admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:07.749647 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1253/2496] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:40:07 2022] PUT /identity/v3/projects/d588b49ea77c4393a6cadf4b0679d8e6/users/cd10a1f4111d4291915d25ad08c1afb9/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:07.771419 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8d7d1-4d48-4f28-aeeb-5a7c9cebb95e None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:07.772773 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8d7d1-4d48-4f28-aeeb-5a7c9cebb95e None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:07.773161 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8d7d1-4d48-4f28-aeeb-5a7c9cebb95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:07.814122 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-a4e8d7d1-4d48-4f28-aeeb-5a7c9cebb95e None None] MFA Rules not processed for user `cd10a1f4111d4291915d25ad08c1afb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:07.815527 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-a4e8d7d1-4d48-4f28-aeeb-5a7c9cebb95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:07.859640 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:07.925537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1244/2497] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:07 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:07.927328 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0917d8-7bb3-4329-ba56-21f0c187046f admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:08.011215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0917d8-7bb3-4329-ba56-21f0c187046f admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:08.036829 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4576de-e93e-46c4-8a94-8a747a90fa0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:08.037796 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4576de-e93e-46c4-8a94-8a747a90fa0d admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:08.038961 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4576de-e93e-46c4-8a94-8a747a90fa0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:08.039249 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4576de-e93e-46c4-8a94-8a747a90fa0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:08.041004 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4576de-e93e-46c4-8a94-8a747a90fa0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:08.042291 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4576de-e93e-46c4-8a94-8a747a90fa0d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:08.068475 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:08.085730 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1254/2498] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:40:07 2022] POST /identity/v3/projects => generated 367 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:40:08.090299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e8d7d1-4d48-4f28-aeeb-5a7c9cebb95e None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:08.104277 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e8d7d1-4d48-4f28-aeeb-5a7c9cebb95e None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:08.180494 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fee2b-efa0-4c0d-81b0-60bd30456062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:08.181273 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fee2b-efa0-4c0d-81b0-60bd30456062 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:08.181661 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fee2b-efa0-4c0d-81b0-60bd30456062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:08.182001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fee2b-efa0-4c0d-81b0-60bd30456062 admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:08.182765 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fee2b-efa0-4c0d-81b0-60bd30456062 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:08.184911 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fee2b-efa0-4c0d-81b0-60bd30456062 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:08.211253 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1245/2499] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:40:08 2022] POST /identity/v3/users => generated 290 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:40:08.215780 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4576de-e93e-46c4-8a94-8a747a90fa0d admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:08.226628 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4576de-e93e-46c4-8a94-8a747a90fa0d admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:08.232108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3a5a2-6db3-480a-b495-b9de7ecc0879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:08.232809 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3a5a2-6db3-480a-b495-b9de7ecc0879 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:08.233229 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3a5a2-6db3-480a-b495-b9de7ecc0879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:08.233519 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3a5a2-6db3-480a-b495-b9de7ecc0879 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:08.234247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3a5a2-6db3-480a-b495-b9de7ecc0879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:08.236011 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3a5a2-6db3-480a-b495-b9de7ecc0879 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:08.248972 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1255/2500] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:40:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:40:08.254657 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fee2b-efa0-4c0d-81b0-60bd30456062 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:08.274150 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fee2b-efa0-4c0d-81b0-60bd30456062 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:08.282037 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e7adb-aa34-42d4-ae9b-050ae3aec8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:08.283660 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e7adb-aa34-42d4-ae9b-050ae3aec8bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:08.284038 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e7adb-aa34-42d4-ae9b-050ae3aec8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:08.284581 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e7adb-aa34-42d4-ae9b-050ae3aec8bd admin admin] PATH_INFO: `/v3/projects/24adf0b92604427ba31d869545148886/users/dac989b241844bcfbda037d5c811dc13/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:08.308957 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e7adb-aa34-42d4-ae9b-050ae3aec8bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=24adf0b92604427ba31d869545148886, user_id=dac989b241844bcfbda037d5c811dc13, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:08.311829 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e7adb-aa34-42d4-ae9b-050ae3aec8bd admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:08.328096 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1246/2501] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:40:08 2022] PUT /identity/v3/projects/24adf0b92604427ba31d869545148886/users/dac989b241844bcfbda037d5c811dc13/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:08.334205 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3a5a2-6db3-480a-b495-b9de7ecc0879 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:08.348040 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3a5a2-6db3-480a-b495-b9de7ecc0879 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:08.413531 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfd84bd-1cbf-405d-8305-baf5f9a4fb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:08.414218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd84bd-1cbf-405d-8305-baf5f9a4fb8e admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:08.414511 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd84bd-1cbf-405d-8305-baf5f9a4fb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:08.414789 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfd84bd-1cbf-405d-8305-baf5f9a4fb8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:08.415521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd84bd-1cbf-405d-8305-baf5f9a4fb8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:08.417252 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfd84bd-1cbf-405d-8305-baf5f9a4fb8e admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:08.429389 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1256/2502] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:40:08 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:40:08.433646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e7adb-aa34-42d4-ae9b-050ae3aec8bd admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:08.445319 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e7adb-aa34-42d4-ae9b-050ae3aec8bd admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:08.450233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14f010-6dab-409f-aa0f-d1b7a5072fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:08.451155 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14f010-6dab-409f-aa0f-d1b7a5072fd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:08.451441 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14f010-6dab-409f-aa0f-d1b7a5072fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:08.451727 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14f010-6dab-409f-aa0f-d1b7a5072fd5 admin admin] PATH_INFO: `/v3/projects/24adf0b92604427ba31d869545148886/users/dac989b241844bcfbda037d5c811dc13/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:08.454054 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f14f010-6dab-409f-aa0f-d1b7a5072fd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24adf0b92604427ba31d869545148886, user_id=dac989b241844bcfbda037d5c811dc13, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:08.456801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f14f010-6dab-409f-aa0f-d1b7a5072fd5 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:08.468692 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1247/2503] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:40:08 2022] PUT /identity/v3/projects/24adf0b92604427ba31d869545148886/users/dac989b241844bcfbda037d5c811dc13/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:08.480806 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12997c4-bae0-45b2-abe6-a7e163833d3b None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:08.481103 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12997c4-bae0-45b2-abe6-a7e163833d3b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:08.481385 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12997c4-bae0-45b2-abe6-a7e163833d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:08.517581 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-d12997c4-bae0-45b2-abe6-a7e163833d3b None None] MFA Rules not processed for user `dac989b241844bcfbda037d5c811dc13`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:08.518841 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d12997c4-bae0-45b2-abe6-a7e163833d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:08.602017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1257/2504] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:08 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:08.616291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14f010-6dab-409f-aa0f-d1b7a5072fd5 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:08.618028 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-1f14f010-6dab-409f-aa0f-d1b7a5072fd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:08.699044 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14f010-6dab-409f-aa0f-d1b7a5072fd5 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:08.719412 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d9d3a-3d28-4129-b3e5-368058ecdf16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:08.720512 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d9d3a-3d28-4129-b3e5-368058ecdf16 service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:08.721358 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d9d3a-3d28-4129-b3e5-368058ecdf16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:08.723411 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d9d3a-3d28-4129-b3e5-368058ecdf16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:08.735222 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7d9d3a-3d28-4129-b3e5-368058ecdf16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:08.737972 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7d9d3a-3d28-4129-b3e5-368058ecdf16 service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:08.766714 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1248/2505] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:40:08 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:40:10.064904 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ada0b7-9eec-410d-9b0b-bfffb13eb66d None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:10.065654 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ada0b7-9eec-410d-9b0b-bfffb13eb66d None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:10.065654 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ada0b7-9eec-410d-9b0b-bfffb13eb66d None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:10.068071 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1258/2506] 10.222.0.16 () {54 vars in 921 bytes} [Mon May 23 12:40:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:40:10.082579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769116e-36d4-462b-bee1-4fc7dc3ed071 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:10.083179 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769116e-36d4-462b-bee1-4fc7dc3ed071 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:10.083179 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769116e-36d4-462b-bee1-4fc7dc3ed071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:10.112052 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-6769116e-36d4-462b-bee1-4fc7dc3ed071 None None] MFA Rules not processed for user `b83535b443a84275a160cd84cb3ad5a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:10.114879 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-6769116e-36d4-462b-bee1-4fc7dc3ed071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:10.137781 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:10.255028 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1249/2507] 10.222.0.16 () {58 vars in 1018 bytes} [Mon May 23 12:40:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:40:10.277126 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ada0b7-9eec-410d-9b0b-bfffb13eb66d None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:10.279786 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-50ada0b7-9eec-410d-9b0b-bfffb13eb66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:10.352621 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ada0b7-9eec-410d-9b0b-bfffb13eb66d None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:10.359506 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c65f8e-ea2a-43f6-8eda-b270d8c1557b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2483a351f5e943ad8c872a98a4717b0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:10.360719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c65f8e-ea2a-43f6-8eda-b270d8c1557b service glance] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:10.361232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c65f8e-ea2a-43f6-8eda-b270d8c1557b service glance] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:10.361727 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c65f8e-ea2a-43f6-8eda-b270d8c1557b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:10.384109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c65f8e-ea2a-43f6-8eda-b270d8c1557b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:10.385949 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c65f8e-ea2a-43f6-8eda-b270d8c1557b service glance] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:10.401731 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1259/2508] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:40:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:40:10.494981 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648c4ec-05f9-4e3f-a316-c8e51c903275 None None] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:10.495288 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648c4ec-05f9-4e3f-a316-c8e51c903275 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:10.495577 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648c4ec-05f9-4e3f-a316-c8e51c903275 None None] PATH_INFO: `/` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:10.497098 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1250/2509] 10.222.0.16 () {54 vars in 921 bytes} [Mon May 23 12:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:40:10.508036 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3e8ce-eaba-4921-a6de-d0eedd69232e None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:10.508036 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3e8ce-eaba-4921-a6de-d0eedd69232e None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:10.508036 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3e8ce-eaba-4921-a6de-d0eedd69232e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:10.524889 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:10.536723 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-09a3e8ce-eaba-4921-a6de-d0eedd69232e None None] MFA Rules not processed for user `b6677be3ddf647e2b971bb3ba2c34a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:10.539690 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-09a3e8ce-eaba-4921-a6de-d0eedd69232e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:10.616734 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1260/2510] 10.222.0.16 () {58 vars in 1018 bytes} [Mon May 23 12:40:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:10.629948 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648c4ec-05f9-4e3f-a316-c8e51c903275 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:10.631646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-d648c4ec-05f9-4e3f-a316-c8e51c903275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:10.647323 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648c4ec-05f9-4e3f-a316-c8e51c903275 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:10.657788 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca3f97-d02b-4e6f-8141-f2099656808e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:10.659064 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca3f97-d02b-4e6f-8141-f2099656808e service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:10.659368 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca3f97-d02b-4e6f-8141-f2099656808e service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:10.659659 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca3f97-d02b-4e6f-8141-f2099656808e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:10.670255 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca3f97-d02b-4e6f-8141-f2099656808e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:10.671858 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caca3f97-d02b-4e6f-8141-f2099656808e service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:10.688022 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1251/2511] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:40:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:40:10.802562 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963659f7-1604-4b6b-9fcf-b832c1ffb1b8 None None] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:10.803089 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963659f7-1604-4b6b-9fcf-b832c1ffb1b8 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:10.803541 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963659f7-1604-4b6b-9fcf-b832c1ffb1b8 None None] PATH_INFO: `/` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:10.805283 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1261/2512] 10.222.0.16 () {54 vars in 921 bytes} [Mon May 23 12:40:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 12:40:10.814640 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de336a09-4ef0-4cad-9028-6d773ccbc537 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:10.815083 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de336a09-4ef0-4cad-9028-6d773ccbc537 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:10.815644 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de336a09-4ef0-4cad-9028-6d773ccbc537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:10.842416 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-de336a09-4ef0-4cad-9028-6d773ccbc537 None None] MFA Rules not processed for user `b83535b443a84275a160cd84cb3ad5a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:10.843661 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-de336a09-4ef0-4cad-9028-6d773ccbc537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:10.857692 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1252/2513] 10.222.0.16 () {58 vars in 1018 bytes} [Mon May 23 12:40:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:10.869049 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963659f7-1604-4b6b-9fcf-b832c1ffb1b8 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:10.871430 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-963659f7-1604-4b6b-9fcf-b832c1ffb1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:10.892346 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963659f7-1604-4b6b-9fcf-b832c1ffb1b8 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:10.901129 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c712dad3-a517-428c-81ad-243054ddfd74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:10.901838 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712dad3-a517-428c-81ad-243054ddfd74 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:10.904662 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712dad3-a517-428c-81ad-243054ddfd74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:10.904662 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712dad3-a517-428c-81ad-243054ddfd74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:10.920536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c712dad3-a517-428c-81ad-243054ddfd74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:10.922481 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c712dad3-a517-428c-81ad-243054ddfd74 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:10.942862 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1262/2514] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:40:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:40:11.024093 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de336a09-4ef0-4cad-9028-6d773ccbc537 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:11.154927 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de336a09-4ef0-4cad-9028-6d773ccbc537 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:11.160592 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f315a078-2179-4a17-af6b-d5a15f3de281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:11.161499 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315a078-2179-4a17-af6b-d5a15f3de281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:11.161766 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315a078-2179-4a17-af6b-d5a15f3de281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:11.162048 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315a078-2179-4a17-af6b-d5a15f3de281 admin admin] PATH_INFO: `/v3/users/cd10a1f4111d4291915d25ad08c1afb9` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:11.163394 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f315a078-2179-4a17-af6b-d5a15f3de281 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd10a1f4111d4291915d25ad08c1afb9)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:11.165387 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f315a078-2179-4a17-af6b-d5a15f3de281 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:11.219131 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f315a078-2179-4a17-af6b-d5a15f3de281 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd10a1f4111d4291915d25ad08c1afb9'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:11.220023 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f315a078-2179-4a17-af6b-d5a15f3de281 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd10a1f4111d4291915d25ad08c1afb9'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:11.231062 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f315a078-2179-4a17-af6b-d5a15f3de281 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd10a1f4111d4291915d25ad08c1afb9'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:11.251095 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f315a078-2179-4a17-af6b-d5a15f3de281 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd10a1f4111d4291915d25ad08c1afb9'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:11.261114 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f315a078-2179-4a17-af6b-d5a15f3de281 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd10a1f4111d4291915d25ad08c1afb9'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:11.268622 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1253/2515] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:40:11 2022] DELETE /identity/v3/users/cd10a1f4111d4291915d25ad08c1afb9 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:11.274442 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe306ae-85b3-4388-8348-fb445937498b None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:11.275160 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe306ae-85b3-4388-8348-fb445937498b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:11.275713 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe306ae-85b3-4388-8348-fb445937498b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:11.306707 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-6fe306ae-85b3-4388-8348-fb445937498b None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:11.309500 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-6fe306ae-85b3-4388-8348-fb445937498b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:11.438473 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1263/2516] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:40:11.458635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f315a078-2179-4a17-af6b-d5a15f3de281 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:11.460718 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-f315a078-2179-4a17-af6b-d5a15f3de281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:11.480684 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:11.561634 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f315a078-2179-4a17-af6b-d5a15f3de281 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:11.566106 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5e2c3-2989-4e68-aec7-edc8d5fec566 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:11.567311 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5e2c3-2989-4e68-aec7-edc8d5fec566 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:11.567791 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5e2c3-2989-4e68-aec7-edc8d5fec566 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:11.568071 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e5e2c3-2989-4e68-aec7-edc8d5fec566 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:11.578544 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e5e2c3-2989-4e68-aec7-edc8d5fec566 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:11.580404 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e5e2c3-2989-4e68-aec7-edc8d5fec566 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:11.593060 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1254/2517] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:40:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:40:11.895749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe306ae-85b3-4388-8348-fb445937498b None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:11.897680 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-6fe306ae-85b3-4388-8348-fb445937498b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:11.918623 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe306ae-85b3-4388-8348-fb445937498b None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:11.924326 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c507d3-a029-4411-964c-b9e1951b9cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:11.925240 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c507d3-a029-4411-964c-b9e1951b9cb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:11.926076 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c507d3-a029-4411-964c-b9e1951b9cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:11.926360 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c507d3-a029-4411-964c-b9e1951b9cb8 admin admin] PATH_INFO: `/v3/projects/d588b49ea77c4393a6cadf4b0679d8e6` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:11.927575 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c507d3-a029-4411-964c-b9e1951b9cb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d588b49ea77c4393a6cadf4b0679d8e6)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:11.929686 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c507d3-a029-4411-964c-b9e1951b9cb8 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:11.983346 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-b8c507d3-a029-4411-964c-b9e1951b9cb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd588b49ea77c4393a6cadf4b0679d8e6', 'request_id': 'req-b8c507d3-a029-4411-964c-b9e1951b9cb8'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:11.989597 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-b8c507d3-a029-4411-964c-b9e1951b9cb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd588b49ea77c4393a6cadf4b0679d8e6', 'request_id': 'req-b8c507d3-a029-4411-964c-b9e1951b9cb8'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:11.996359 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-b8c507d3-a029-4411-964c-b9e1951b9cb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd588b49ea77c4393a6cadf4b0679d8e6', 'request_id': 'req-b8c507d3-a029-4411-964c-b9e1951b9cb8'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:12.015916 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-b8c507d3-a029-4411-964c-b9e1951b9cb8 admin admin] The token cache is being invalidate because project d588b49ea77c4393a6cadf4b0679d8e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:40:12.016249 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-b8c507d3-a029-4411-964c-b9e1951b9cb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:12.016931 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-b8c507d3-a029-4411-964c-b9e1951b9cb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:12.019886 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1264/2518] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:40:11 2022] DELETE /identity/v3/projects/d588b49ea77c4393a6cadf4b0679d8e6 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:12.029275 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5e2c3-2989-4e68-aec7-edc8d5fec566 service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:12.043409 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e5e2c3-2989-4e68-aec7-edc8d5fec566 service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:12.111558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6024a-f2e2-487c-b6af-daaa134ca8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:12.113013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6024a-f2e2-487c-b6af-daaa134ca8d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:12.113977 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6024a-f2e2-487c-b6af-daaa134ca8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:12.114355 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6024a-f2e2-487c-b6af-daaa134ca8d2 admin admin] PATH_INFO: `/v3/users/dac989b241844bcfbda037d5c811dc13` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:12.115905 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb6024a-f2e2-487c-b6af-daaa134ca8d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dac989b241844bcfbda037d5c811dc13)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:12.118366 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb6024a-f2e2-487c-b6af-daaa134ca8d2 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:12.199127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-9fb6024a-f2e2-487c-b6af-daaa134ca8d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dac989b241844bcfbda037d5c811dc13'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:12.200896 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-9fb6024a-f2e2-487c-b6af-daaa134ca8d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dac989b241844bcfbda037d5c811dc13'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:12.212026 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-9fb6024a-f2e2-487c-b6af-daaa134ca8d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dac989b241844bcfbda037d5c811dc13'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:12.232919 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-9fb6024a-f2e2-487c-b6af-daaa134ca8d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dac989b241844bcfbda037d5c811dc13'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:12.244082 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-9fb6024a-f2e2-487c-b6af-daaa134ca8d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dac989b241844bcfbda037d5c811dc13'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:12.251702 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1255/2519] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:40:12 2022] DELETE /identity/v3/users/dac989b241844bcfbda037d5c811dc13 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:12.807461 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c507d3-a029-4411-964c-b9e1951b9cb8 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:12.809758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-b8c507d3-a029-4411-964c-b9e1951b9cb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:12.825138 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c507d3-a029-4411-964c-b9e1951b9cb8 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:12.874327 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:12.937230 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29cc137-c064-4200-bfcb-0684a04f2932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:12.938116 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cc137-c064-4200-bfcb-0684a04f2932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:12.938400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cc137-c064-4200-bfcb-0684a04f2932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:12.938677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cc137-c064-4200-bfcb-0684a04f2932 admin admin] PATH_INFO: `/v3/projects/24adf0b92604427ba31d869545148886` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:12.940021 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29cc137-c064-4200-bfcb-0684a04f2932 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24adf0b92604427ba31d869545148886)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:12.943834 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29cc137-c064-4200-bfcb-0684a04f2932 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:13.008962 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d29cc137-c064-4200-bfcb-0684a04f2932 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24adf0b92604427ba31d869545148886', 'request_id': 'req-d29cc137-c064-4200-bfcb-0684a04f2932'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:13.016836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d29cc137-c064-4200-bfcb-0684a04f2932 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24adf0b92604427ba31d869545148886', 'request_id': 'req-d29cc137-c064-4200-bfcb-0684a04f2932'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:13.025666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d29cc137-c064-4200-bfcb-0684a04f2932 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24adf0b92604427ba31d869545148886', 'request_id': 'req-d29cc137-c064-4200-bfcb-0684a04f2932'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:13.039560 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d29cc137-c064-4200-bfcb-0684a04f2932 admin admin] The token cache is being invalidate because project 24adf0b92604427ba31d869545148886 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:40:13.039852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d29cc137-c064-4200-bfcb-0684a04f2932 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:13.040499 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d29cc137-c064-4200-bfcb-0684a04f2932 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:13.042222 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1265/2520] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:40:12 2022] DELETE /identity/v3/projects/24adf0b92604427ba31d869545148886 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:13.047527 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860e8ca-993e-4bb4-9013-2312b961ed17 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:13.047732 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860e8ca-993e-4bb4-9013-2312b961ed17 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:13.048003 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860e8ca-993e-4bb4-9013-2312b961ed17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:13.067935 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-7860e8ca-993e-4bb4-9013-2312b961ed17 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:13.069248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-7860e8ca-993e-4bb4-9013-2312b961ed17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:13.108729 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:13.152294 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1256/2521] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:13.159768 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b3ae4-1059-4148-88e8-bbe84880d4b7 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:13.160047 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b3ae4-1059-4148-88e8-bbe84880d4b7 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:13.160425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b3ae4-1059-4148-88e8-bbe84880d4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:13.182182 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-002b3ae4-1059-4148-88e8-bbe84880d4b7 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:13.183288 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-002b3ae4-1059-4148-88e8-bbe84880d4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:13.189642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4777dd-5548-42ad-b5b9-13ca57b1120d None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:13.189937 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4777dd-5548-42ad-b5b9-13ca57b1120d None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:13.190224 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4777dd-5548-42ad-b5b9-13ca57b1120d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:13.195042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1266/2522] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:13.196134 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b3ae4-1059-4148-88e8-bbe84880d4b7 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:13.206511 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b3ae4-1059-4148-88e8-bbe84880d4b7 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:13.211824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1c74c-c034-450b-a70f-2472d6873dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:13.212438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1c74c-c034-450b-a70f-2472d6873dec admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:13.212721 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1c74c-c034-450b-a70f-2472d6873dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:13.212998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1c74c-c034-450b-a70f-2472d6873dec admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:13.213784 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1c74c-c034-450b-a70f-2472d6873dec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:40:13.214181 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1c74c-c034-450b-a70f-2472d6873dec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:13.214875 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-9e4777dd-5548-42ad-b5b9-13ca57b1120d None None] MFA Rules not processed for user `733f8296ec3d4795a1140c1e1cb6eb95`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:13.215683 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1c74c-c034-450b-a70f-2472d6873dec admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:13.216106 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-9e4777dd-5548-42ad-b5b9-13ca57b1120d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:13.228724 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1267/2523] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:40:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:40:13.233556 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1c74c-c034-450b-a70f-2472d6873dec admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:13.246379 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1c74c-c034-450b-a70f-2472d6873dec admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:13.251676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b7303-55db-4962-8eac-3511b88490c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:13.252445 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b7303-55db-4962-8eac-3511b88490c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:13.252824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b7303-55db-4962-8eac-3511b88490c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:13.253173 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b7303-55db-4962-8eac-3511b88490c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:13.254530 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b7303-55db-4962-8eac-3511b88490c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:13.256448 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b7303-55db-4962-8eac-3511b88490c7 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:13.281364 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1257/2524] 10.222.0.16 () {58 vars in 1009 bytes} [Mon May 23 12:40:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:40:13.292215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1268/2525] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:40:13 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:40:13.295300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b7303-55db-4962-8eac-3511b88490c7 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:13.295516 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4777dd-5548-42ad-b5b9-13ca57b1120d None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:13.296662 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-9e4777dd-5548-42ad-b5b9-13ca57b1120d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:13.370082 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b7303-55db-4962-8eac-3511b88490c7 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:13.374949 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4777dd-5548-42ad-b5b9-13ca57b1120d None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:13.375959 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e617b-5048-459e-b861-ecc6c8b906fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:13.376886 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e617b-5048-459e-b861-ecc6c8b906fa admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:13.377226 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e617b-5048-459e-b861-ecc6c8b906fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:13.377543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e617b-5048-459e-b861-ecc6c8b906fa admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:13.378320 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e617b-5048-459e-b861-ecc6c8b906fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:13.381138 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92e617b-5048-459e-b861-ecc6c8b906fa admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:13.409573 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1269/2526] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:40:13 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:40:13.414119 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e617b-5048-459e-b861-ecc6c8b906fa admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:13.425257 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:13.431039 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e617b-5048-459e-b861-ecc6c8b906fa admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:13.437555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d46907-d2a7-4b43-9d90-32f95c5ed3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:13.438222 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d46907-d2a7-4b43-9d90-32f95c5ed3b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:13.438501 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d46907-d2a7-4b43-9d90-32f95c5ed3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:13.439121 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d46907-d2a7-4b43-9d90-32f95c5ed3b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:13.439524 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d46907-d2a7-4b43-9d90-32f95c5ed3b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:13.442430 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d46907-d2a7-4b43-9d90-32f95c5ed3b2 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:13.459213 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1270/2527] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:40:13.465400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d46907-d2a7-4b43-9d90-32f95c5ed3b2 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:13.481482 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d46907-d2a7-4b43-9d90-32f95c5ed3b2 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:13.487740 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee5203-941e-4f0e-aed3-34a5cf7f8a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:13.489253 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee5203-941e-4f0e-aed3-34a5cf7f8a21 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:13.489545 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee5203-941e-4f0e-aed3-34a5cf7f8a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:13.489823 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee5203-941e-4f0e-aed3-34a5cf7f8a21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:13.490563 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee5203-941e-4f0e-aed3-34a5cf7f8a21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:13.492197 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee5203-941e-4f0e-aed3-34a5cf7f8a21 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:13.494348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119bceb-c594-4cb9-83c0-94e0fdf2ee9e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71fb9a79ac494635804b93d4a0048205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:13.495436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119bceb-c594-4cb9-83c0-94e0fdf2ee9e service swift] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:13.495909 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119bceb-c594-4cb9-83c0-94e0fdf2ee9e service swift] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:13.496406 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119bceb-c594-4cb9-83c0-94e0fdf2ee9e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:13.514061 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1271/2528] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:40:13.515201 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee5203-941e-4f0e-aed3-34a5cf7f8a21 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:13.529599 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee5203-941e-4f0e-aed3-34a5cf7f8a21 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:13.535701 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:13.536144 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e643060f-8bf6-4fa4-873d-3cc08c1ce726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:13.537080 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643060f-8bf6-4fa4-873d-3cc08c1ce726 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:13.537361 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643060f-8bf6-4fa4-873d-3cc08c1ce726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:13.537643 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e643060f-8bf6-4fa4-873d-3cc08c1ce726 admin admin] PATH_INFO: `/v3/projects/b8f5a51eab924caf8350bbb375e381e7/users/abcc498119f245f58ce791f3288f0652/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:13.560107 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e643060f-8bf6-4fa4-873d-3cc08c1ce726 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8f5a51eab924caf8350bbb375e381e7, user_id=abcc498119f245f58ce791f3288f0652, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:13.563397 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e643060f-8bf6-4fa4-873d-3cc08c1ce726 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:13.583073 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1272/2529] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:40:13 2022] PUT /identity/v3/projects/b8f5a51eab924caf8350bbb375e381e7/users/abcc498119f245f58ce791f3288f0652/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:13.588842 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119bceb-c594-4cb9-83c0-94e0fdf2ee9e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:13.590457 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119bceb-c594-4cb9-83c0-94e0fdf2ee9e service swift] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:13.597391 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e061e3-8179-46ad-bebc-81109dc3369f None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:13.597841 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e061e3-8179-46ad-bebc-81109dc3369f None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:13.598150 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e061e3-8179-46ad-bebc-81109dc3369f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:13.600550 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1258/2530] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 12:40:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 12:40:13.638124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-05e061e3-8179-46ad-bebc-81109dc3369f None None] MFA Rules not processed for user `abcc498119f245f58ce791f3288f0652`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:13.639824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-05e061e3-8179-46ad-bebc-81109dc3369f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:13.827676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1273/2531] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:13.836285 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119bceb-c594-4cb9-83c0-94e0fdf2ee9e service swift] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:13.890417 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119bceb-c594-4cb9-83c0-94e0fdf2ee9e service swift] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:14.094656 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f4e25-02ad-432d-a0dc-88b8c16896ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:14.097786 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f4e25-02ad-432d-a0dc-88b8c16896ad admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:14.098577 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f4e25-02ad-432d-a0dc-88b8c16896ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:14.099122 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f4e25-02ad-432d-a0dc-88b8c16896ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:14.099989 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595f4e25-02ad-432d-a0dc-88b8c16896ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:14.102233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595f4e25-02ad-432d-a0dc-88b8c16896ad admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:14.149635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1259/2532] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:40:13 2022] POST /identity/v3/projects => generated 385 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:40:14.155220 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e061e3-8179-46ad-bebc-81109dc3369f None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:14.169244 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e061e3-8179-46ad-bebc-81109dc3369f None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:14.313576 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee7490-6c40-444c-b233-8666118ddb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:14.315020 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee7490-6c40-444c-b233-8666118ddb91 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:14.315793 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee7490-6c40-444c-b233-8666118ddb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:14.316629 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee7490-6c40-444c-b233-8666118ddb91 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:14.318052 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee7490-6c40-444c-b233-8666118ddb91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:14.322364 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee7490-6c40-444c-b233-8666118ddb91 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:14.377919 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1274/2533] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:40:14 2022] POST /identity/v3/users => generated 299 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:40:14.381165 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f4e25-02ad-432d-a0dc-88b8c16896ad admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:14.404223 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f4e25-02ad-432d-a0dc-88b8c16896ad admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:14.413523 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf51eb67-63bd-495b-8e21-8b370835951b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:14.414742 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf51eb67-63bd-495b-8e21-8b370835951b admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:14.415149 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf51eb67-63bd-495b-8e21-8b370835951b admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:14.415633 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf51eb67-63bd-495b-8e21-8b370835951b admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:14.419348 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf51eb67-63bd-495b-8e21-8b370835951b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:14.422671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf51eb67-63bd-495b-8e21-8b370835951b admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:14.450337 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1260/2534] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:40:14.452253 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee7490-6c40-444c-b233-8666118ddb91 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:14.470153 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee7490-6c40-444c-b233-8666118ddb91 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:14.475907 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f529a84-538a-4ee1-a1b7-d01bd99ed12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:14.477150 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f529a84-538a-4ee1-a1b7-d01bd99ed12e admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:14.477620 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f529a84-538a-4ee1-a1b7-d01bd99ed12e admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:14.478081 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f529a84-538a-4ee1-a1b7-d01bd99ed12e admin admin] PATH_INFO: `/v3/projects/f3aa51004fe8427c98a62e128df6620a/users/7029fcf1245343d3992db7b9d7bb6e6c/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:14.513090 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f529a84-538a-4ee1-a1b7-d01bd99ed12e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3aa51004fe8427c98a62e128df6620a, user_id=7029fcf1245343d3992db7b9d7bb6e6c, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:14.516841 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f529a84-538a-4ee1-a1b7-d01bd99ed12e admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:14.555134 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1275/2535] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:40:14 2022] PUT /identity/v3/projects/f3aa51004fe8427c98a62e128df6620a/users/7029fcf1245343d3992db7b9d7bb6e6c/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:14.556152 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf51eb67-63bd-495b-8e21-8b370835951b admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:14.572646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf51eb67-63bd-495b-8e21-8b370835951b admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:14.735781 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d03a6c5-9e39-43cd-aea8-495afc1650b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:14.736503 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03a6c5-9e39-43cd-aea8-495afc1650b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:14.736797 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03a6c5-9e39-43cd-aea8-495afc1650b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:14.737157 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d03a6c5-9e39-43cd-aea8-495afc1650b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:14.738807 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03a6c5-9e39-43cd-aea8-495afc1650b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:14.740694 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d03a6c5-9e39-43cd-aea8-495afc1650b5 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:14.760335 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1261/2536] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:40:14.765317 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f529a84-538a-4ee1-a1b7-d01bd99ed12e admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:14.785751 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f529a84-538a-4ee1-a1b7-d01bd99ed12e admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:14.791964 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4934f5b-bceb-47da-a52d-ff4b92d05f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:14.793125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4934f5b-bceb-47da-a52d-ff4b92d05f37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:14.793468 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4934f5b-bceb-47da-a52d-ff4b92d05f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:14.793811 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4934f5b-bceb-47da-a52d-ff4b92d05f37 admin admin] PATH_INFO: `/v3/projects/f3aa51004fe8427c98a62e128df6620a/users/7029fcf1245343d3992db7b9d7bb6e6c/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:14.796396 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4934f5b-bceb-47da-a52d-ff4b92d05f37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3aa51004fe8427c98a62e128df6620a, user_id=7029fcf1245343d3992db7b9d7bb6e6c, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:14.801269 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4934f5b-bceb-47da-a52d-ff4b92d05f37 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:14.822799 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:14.837855 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1276/2537] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:40:14 2022] PUT /identity/v3/projects/f3aa51004fe8427c98a62e128df6620a/users/7029fcf1245343d3992db7b9d7bb6e6c/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:14.850133 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbef292-6b83-42e1-ad8a-e0f8d27f815e None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:14.850431 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbef292-6b83-42e1-ad8a-e0f8d27f815e None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:14.850722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbef292-6b83-42e1-ad8a-e0f8d27f815e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:14.914463 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-9fbef292-6b83-42e1-ad8a-e0f8d27f815e None None] MFA Rules not processed for user `7029fcf1245343d3992db7b9d7bb6e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:14.917616 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-9fbef292-6b83-42e1-ad8a-e0f8d27f815e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:14.959844 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:15.081142 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1262/2538] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:15.091869 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4934f5b-bceb-47da-a52d-ff4b92d05f37 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:15.095387 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e4934f5b-bceb-47da-a52d-ff4b92d05f37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:15.219815 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4934f5b-bceb-47da-a52d-ff4b92d05f37 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:15.224158 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb785b9d-5594-4255-b13b-41754125c576 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:15.224968 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb785b9d-5594-4255-b13b-41754125c576 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:15.225325 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb785b9d-5594-4255-b13b-41754125c576 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:15.225760 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb785b9d-5594-4255-b13b-41754125c576 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:15.237168 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb785b9d-5594-4255-b13b-41754125c576 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:15.239925 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb785b9d-5594-4255-b13b-41754125c576 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:15.252075 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1277/2539] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:40:15 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:40:15.347993 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbef292-6b83-42e1-ad8a-e0f8d27f815e None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:15.438613 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbef292-6b83-42e1-ad8a-e0f8d27f815e None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:15.447794 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b5984-206b-4520-8969-99e29d757d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:15.450488 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b5984-206b-4520-8969-99e29d757d16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:15.451232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b5984-206b-4520-8969-99e29d757d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:15.451846 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7b5984-206b-4520-8969-99e29d757d16 admin admin] PATH_INFO: `/v3/users/abcc498119f245f58ce791f3288f0652` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:15.455390 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b5984-206b-4520-8969-99e29d757d16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abcc498119f245f58ce791f3288f0652)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:15.460480 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7b5984-206b-4520-8969-99e29d757d16 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:15.557478 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-4f7b5984-206b-4520-8969-99e29d757d16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abcc498119f245f58ce791f3288f0652'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:15.558333 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-4f7b5984-206b-4520-8969-99e29d757d16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abcc498119f245f58ce791f3288f0652'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:15.567767 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-4f7b5984-206b-4520-8969-99e29d757d16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abcc498119f245f58ce791f3288f0652'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:15.586387 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-4f7b5984-206b-4520-8969-99e29d757d16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abcc498119f245f58ce791f3288f0652'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:15.604254 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-4f7b5984-206b-4520-8969-99e29d757d16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abcc498119f245f58ce791f3288f0652'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:15.618866 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1263/2540] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:40:15 2022] DELETE /identity/v3/users/abcc498119f245f58ce791f3288f0652 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:15.623864 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48ae5c-055c-4a80-9ff6-9b9496c627a7 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:15.624131 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48ae5c-055c-4a80-9ff6-9b9496c627a7 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:15.624797 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48ae5c-055c-4a80-9ff6-9b9496c627a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:15.691241 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-6a48ae5c-055c-4a80-9ff6-9b9496c627a7 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:15.692652 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-6a48ae5c-055c-4a80-9ff6-9b9496c627a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:15.842232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1278/2541] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:15.858953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b5984-206b-4520-8969-99e29d757d16 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:15.860803 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-4f7b5984-206b-4520-8969-99e29d757d16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:15.951819 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:15.992220 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7b5984-206b-4520-8969-99e29d757d16 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:15.996478 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad34aa-52c6-40ad-aad5-a02b6cf0e688 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:15.997382 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad34aa-52c6-40ad-aad5-a02b6cf0e688 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:15.997678 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad34aa-52c6-40ad-aad5-a02b6cf0e688 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:15.998109 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad34aa-52c6-40ad-aad5-a02b6cf0e688 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:16.008991 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ad34aa-52c6-40ad-aad5-a02b6cf0e688 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:16.010882 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ad34aa-52c6-40ad-aad5-a02b6cf0e688 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:16.029279 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1264/2542] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:40:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:40:16.434577 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a48ae5c-055c-4a80-9ff6-9b9496c627a7 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:16.436586 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-6a48ae5c-055c-4a80-9ff6-9b9496c627a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:16.450620 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a48ae5c-055c-4a80-9ff6-9b9496c627a7 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:16.457306 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb92842-294f-40d5-8c13-dea726258aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:16.458762 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb92842-294f-40d5-8c13-dea726258aba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:16.458762 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb92842-294f-40d5-8c13-dea726258aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:16.458762 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb92842-294f-40d5-8c13-dea726258aba admin admin] PATH_INFO: `/v3/projects/b8f5a51eab924caf8350bbb375e381e7` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:16.459949 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb92842-294f-40d5-8c13-dea726258aba admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8f5a51eab924caf8350bbb375e381e7)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:16.462135 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb92842-294f-40d5-8c13-dea726258aba admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:16.548421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3eb92842-294f-40d5-8c13-dea726258aba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8f5a51eab924caf8350bbb375e381e7', 'request_id': 'req-3eb92842-294f-40d5-8c13-dea726258aba'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:16.555245 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3eb92842-294f-40d5-8c13-dea726258aba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8f5a51eab924caf8350bbb375e381e7', 'request_id': 'req-3eb92842-294f-40d5-8c13-dea726258aba'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:16.564554 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3eb92842-294f-40d5-8c13-dea726258aba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8f5a51eab924caf8350bbb375e381e7', 'request_id': 'req-3eb92842-294f-40d5-8c13-dea726258aba'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:16.585384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3eb92842-294f-40d5-8c13-dea726258aba admin admin] The token cache is being invalidate because project b8f5a51eab924caf8350bbb375e381e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:40:16.586059 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3eb92842-294f-40d5-8c13-dea726258aba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:16.586998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3eb92842-294f-40d5-8c13-dea726258aba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:16.589507 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1279/2543] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:40:16 2022] DELETE /identity/v3/projects/b8f5a51eab924caf8350bbb375e381e7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:16.598201 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad34aa-52c6-40ad-aad5-a02b6cf0e688 service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:16.608726 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad34aa-52c6-40ad-aad5-a02b6cf0e688 service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:16.704917 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92bd37-cd23-484d-96c6-ebb868c0f47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:16.706363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92bd37-cd23-484d-96c6-ebb868c0f47a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:16.706567 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92bd37-cd23-484d-96c6-ebb868c0f47a admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:16.706846 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92bd37-cd23-484d-96c6-ebb868c0f47a admin admin] PATH_INFO: `/v3/users/7029fcf1245343d3992db7b9d7bb6e6c` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:16.708287 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be92bd37-cd23-484d-96c6-ebb868c0f47a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7029fcf1245343d3992db7b9d7bb6e6c)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:16.710233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be92bd37-cd23-484d-96c6-ebb868c0f47a admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:16.807886 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-be92bd37-cd23-484d-96c6-ebb868c0f47a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7029fcf1245343d3992db7b9d7bb6e6c'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:16.808832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-be92bd37-cd23-484d-96c6-ebb868c0f47a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7029fcf1245343d3992db7b9d7bb6e6c'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:16.822126 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-be92bd37-cd23-484d-96c6-ebb868c0f47a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7029fcf1245343d3992db7b9d7bb6e6c'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:16.844831 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-be92bd37-cd23-484d-96c6-ebb868c0f47a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7029fcf1245343d3992db7b9d7bb6e6c'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:16.857033 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-be92bd37-cd23-484d-96c6-ebb868c0f47a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7029fcf1245343d3992db7b9d7bb6e6c'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:16.866226 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1265/2544] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:40:16 2022] DELETE /identity/v3/users/7029fcf1245343d3992db7b9d7bb6e6c => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:17.116586 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb92842-294f-40d5-8c13-dea726258aba admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:17.118704 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3eb92842-294f-40d5-8c13-dea726258aba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:17.138803 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb92842-294f-40d5-8c13-dea726258aba admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:17.194375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:17.273720 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1215e422-1fc4-4906-bbc0-ed9c279df1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:17.274627 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215e422-1fc4-4906-bbc0-ed9c279df1f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:17.274913 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215e422-1fc4-4906-bbc0-ed9c279df1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:17.275192 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215e422-1fc4-4906-bbc0-ed9c279df1f2 admin admin] PATH_INFO: `/v3/users/30b7bb34b99c4cb48e2096808d39ce7a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:17.277546 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1215e422-1fc4-4906-bbc0-ed9c279df1f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30b7bb34b99c4cb48e2096808d39ce7a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:17.279504 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1215e422-1fc4-4906-bbc0-ed9c279df1f2 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:17.478613 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1215e422-1fc4-4906-bbc0-ed9c279df1f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30b7bb34b99c4cb48e2096808d39ce7a'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:17.489158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92bd37-cd23-484d-96c6-ebb868c0f47a admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:17.490867 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-be92bd37-cd23-484d-96c6-ebb868c0f47a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:17.498830 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1215e422-1fc4-4906-bbc0-ed9c279df1f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30b7bb34b99c4cb48e2096808d39ce7a'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:17.514679 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92bd37-cd23-484d-96c6-ebb868c0f47a admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:17.521446 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f8e442-ea21-4aa5-bf04-8371da8f13be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:17.522479 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8e442-ea21-4aa5-bf04-8371da8f13be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:17.522860 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8e442-ea21-4aa5-bf04-8371da8f13be admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:17.523257 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f8e442-ea21-4aa5-bf04-8371da8f13be admin admin] PATH_INFO: `/v3/projects/f3aa51004fe8427c98a62e128df6620a` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:17.525694 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f8e442-ea21-4aa5-bf04-8371da8f13be admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3aa51004fe8427c98a62e128df6620a)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:17.528981 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f8e442-ea21-4aa5-bf04-8371da8f13be admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:17.539222 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1215e422-1fc4-4906-bbc0-ed9c279df1f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30b7bb34b99c4cb48e2096808d39ce7a'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:17.540405 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1215e422-1fc4-4906-bbc0-ed9c279df1f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30b7bb34b99c4cb48e2096808d39ce7a'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:17.541138 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1215e422-1fc4-4906-bbc0-ed9c279df1f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30b7bb34b99c4cb48e2096808d39ce7a'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:17.558289 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1280/2545] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:40:17 2022] DELETE /identity/v3/users/30b7bb34b99c4cb48e2096808d39ce7a => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:17.563197 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7754b5-544e-49bb-b9a2-9ea8bbbae8c6 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:17.563600 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7754b5-544e-49bb-b9a2-9ea8bbbae8c6 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:17.563988 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7754b5-544e-49bb-b9a2-9ea8bbbae8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:17.589873 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-ba7754b5-544e-49bb-b9a2-9ea8bbbae8c6 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:17.591945 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ba7754b5-544e-49bb-b9a2-9ea8bbbae8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:17.635600 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-40f8e442-ea21-4aa5-bf04-8371da8f13be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3aa51004fe8427c98a62e128df6620a', 'request_id': 'req-40f8e442-ea21-4aa5-bf04-8371da8f13be'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:17.638889 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:17.648108 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-40f8e442-ea21-4aa5-bf04-8371da8f13be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3aa51004fe8427c98a62e128df6620a', 'request_id': 'req-40f8e442-ea21-4aa5-bf04-8371da8f13be'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:17.665760 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-40f8e442-ea21-4aa5-bf04-8371da8f13be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3aa51004fe8427c98a62e128df6620a', 'request_id': 'req-40f8e442-ea21-4aa5-bf04-8371da8f13be'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:17.696929 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-40f8e442-ea21-4aa5-bf04-8371da8f13be admin admin] The token cache is being invalidate because project f3aa51004fe8427c98a62e128df6620a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:40:17.697396 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-40f8e442-ea21-4aa5-bf04-8371da8f13be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:17.698225 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-40f8e442-ea21-4aa5-bf04-8371da8f13be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:17.699883 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1281/2546] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:17.700086 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1266/2547] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:40:17 2022] DELETE /identity/v3/projects/f3aa51004fe8427c98a62e128df6620a => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:40:17.706995 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519c65f-324d-4cc3-9740-a90de3a34bdd None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:17.707999 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519c65f-324d-4cc3-9740-a90de3a34bdd None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:17.708783 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519c65f-324d-4cc3-9740-a90de3a34bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:17.723140 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f8e442-ea21-4aa5-bf04-8371da8f13be admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:17.725614 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-40f8e442-ea21-4aa5-bf04-8371da8f13be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:17.754772 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-a519c65f-324d-4cc3-9740-a90de3a34bdd None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:17.756942 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a519c65f-324d-4cc3-9740-a90de3a34bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:17.812564 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:17.822278 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f8e442-ea21-4aa5-bf04-8371da8f13be admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:17.829761 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f090f7-670c-40fa-ae5c-d2ebc62cb458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:17.830986 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f090f7-670c-40fa-ae5c-d2ebc62cb458 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:17.832373 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f090f7-670c-40fa-ae5c-d2ebc62cb458 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:17.832976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f090f7-670c-40fa-ae5c-d2ebc62cb458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:17.836645 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-75f090f7-670c-40fa-ae5c-d2ebc62cb458 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:17.862504 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1282/2548] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:17.870239 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41313773-cc6a-4de0-a711-0cfc296071d7 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:17.870530 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41313773-cc6a-4de0-a711-0cfc296071d7 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:17.870812 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41313773-cc6a-4de0-a711-0cfc296071d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:17.916396 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-41313773-cc6a-4de0-a711-0cfc296071d7 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:17.917767 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-41313773-cc6a-4de0-a711-0cfc296071d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:17.930151 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1283/2549] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:17.939390 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41313773-cc6a-4de0-a711-0cfc296071d7 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:17.950844 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41313773-cc6a-4de0-a711-0cfc296071d7 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:17.966892 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfe55f-00df-42a7-94db-eae8ee80043f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:17.968407 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfe55f-00df-42a7-94db-eae8ee80043f admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:17.969328 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfe55f-00df-42a7-94db-eae8ee80043f admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:17.970234 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfe55f-00df-42a7-94db-eae8ee80043f admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:17.971932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dfe55f-00df-42a7-94db-eae8ee80043f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:40:17.972931 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dfe55f-00df-42a7-94db-eae8ee80043f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:17.982788 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dfe55f-00df-42a7-94db-eae8ee80043f admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:17.994374 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f090f7-670c-40fa-ae5c-d2ebc62cb458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:17.997292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f090f7-670c-40fa-ae5c-d2ebc62cb458 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:18.008039 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1284/2550] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:40:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:40:18.014421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfe55f-00df-42a7-94db-eae8ee80043f admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:18.028798 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1267/2551] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:40:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:40:18.029865 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfe55f-00df-42a7-94db-eae8ee80043f admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:18.040268 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80be13e-3bea-42b7-8d67-2a6c80607eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:18.041530 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80be13e-3bea-42b7-8d67-2a6c80607eff admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:18.042101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80be13e-3bea-42b7-8d67-2a6c80607eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:18.043052 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80be13e-3bea-42b7-8d67-2a6c80607eff admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:18.043570 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80be13e-3bea-42b7-8d67-2a6c80607eff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:18.055332 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80be13e-3bea-42b7-8d67-2a6c80607eff admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:18.103104 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1285/2552] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:40:18 2022] POST /identity/v3/projects => generated 385 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:40:18.109322 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f090f7-670c-40fa-ae5c-d2ebc62cb458 service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:18.120833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f090f7-670c-40fa-ae5c-d2ebc62cb458 service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:18.203072 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc6111-1f47-43e3-9ca5-1a9e37260b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:18.204029 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc6111-1f47-43e3-9ca5-1a9e37260b1a admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:18.204386 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc6111-1f47-43e3-9ca5-1a9e37260b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:18.204643 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc6111-1f47-43e3-9ca5-1a9e37260b1a admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:18.205368 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc6111-1f47-43e3-9ca5-1a9e37260b1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:18.208288 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc6111-1f47-43e3-9ca5-1a9e37260b1a admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:18.244291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1268/2553] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:40:18 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:40:18.245502 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80be13e-3bea-42b7-8d67-2a6c80607eff admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:18.266807 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80be13e-3bea-42b7-8d67-2a6c80607eff admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:18.272835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad37a93-408f-4b86-b257-247e0c1cc1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:18.273678 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad37a93-408f-4b86-b257-247e0c1cc1fa admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:18.273969 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad37a93-408f-4b86-b257-247e0c1cc1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:18.274433 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad37a93-408f-4b86-b257-247e0c1cc1fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:18.275179 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad37a93-408f-4b86-b257-247e0c1cc1fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:18.277092 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad37a93-408f-4b86-b257-247e0c1cc1fa admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:18.288630 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1286/2554] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:40:18.293317 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc6111-1f47-43e3-9ca5-1a9e37260b1a admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:18.305624 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc6111-1f47-43e3-9ca5-1a9e37260b1a admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:18.312341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e25859-32cd-4b66-940b-0a8b134b1c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:18.313125 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e25859-32cd-4b66-940b-0a8b134b1c19 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:18.313494 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e25859-32cd-4b66-940b-0a8b134b1c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:18.313816 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e25859-32cd-4b66-940b-0a8b134b1c19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:18.314879 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e25859-32cd-4b66-940b-0a8b134b1c19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:18.317973 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e25859-32cd-4b66-940b-0a8b134b1c19 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:18.342076 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1269/2555] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:40:18.346864 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad37a93-408f-4b86-b257-247e0c1cc1fa admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:18.358768 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad37a93-408f-4b86-b257-247e0c1cc1fa admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:18.365006 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d04007-8f2b-4bd8-b49c-c8e401c8a758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:18.366058 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d04007-8f2b-4bd8-b49c-c8e401c8a758 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:18.366350 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d04007-8f2b-4bd8-b49c-c8e401c8a758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:18.366633 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d04007-8f2b-4bd8-b49c-c8e401c8a758 admin admin] PATH_INFO: `/v3/projects/c7193dfd80704d9ab9b4c0ded7c938d1/users/f586f39e308847aebced04e3d149656c/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:18.387850 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d04007-8f2b-4bd8-b49c-c8e401c8a758 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7193dfd80704d9ab9b4c0ded7c938d1, user_id=f586f39e308847aebced04e3d149656c, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:18.391089 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d04007-8f2b-4bd8-b49c-c8e401c8a758 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:18.416468 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1287/2556] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:40:18 2022] PUT /identity/v3/projects/c7193dfd80704d9ab9b4c0ded7c938d1/users/f586f39e308847aebced04e3d149656c/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:40:18.436213 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe165b3-9a0b-403c-8c0b-881290a26e7a None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:18.436548 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe165b3-9a0b-403c-8c0b-881290a26e7a None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:18.436854 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe165b3-9a0b-403c-8c0b-881290a26e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:18.487082 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:18.512728 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-6fe165b3-9a0b-403c-8c0b-881290a26e7a None None] MFA Rules not processed for user `f586f39e308847aebced04e3d149656c`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:18.514374 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-6fe165b3-9a0b-403c-8c0b-881290a26e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:18.593757 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d04007-8f2b-4bd8-b49c-c8e401c8a758 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:18.597557 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-71d04007-8f2b-4bd8-b49c-c8e401c8a758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:18.683895 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1270/2557] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:18.689476 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe165b3-9a0b-403c-8c0b-881290a26e7a None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:18.763272 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d04007-8f2b-4bd8-b49c-c8e401c8a758 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:18.778752 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f5aec-395c-45e5-ad57-a570f44d9fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:18.780558 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f5aec-395c-45e5-ad57-a570f44d9fa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:18.782267 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f5aec-395c-45e5-ad57-a570f44d9fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:18.783448 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67f5aec-395c-45e5-ad57-a570f44d9fa0 admin admin] PATH_INFO: `/v3/projects/347d9d75c5064f9fa5b7c6528d464688` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:18.789252 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67f5aec-395c-45e5-ad57-a570f44d9fa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=347d9d75c5064f9fa5b7c6528d464688)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:18.797426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67f5aec-395c-45e5-ad57-a570f44d9fa0 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:18.841975 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe165b3-9a0b-403c-8c0b-881290a26e7a None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:18.876687 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c67f5aec-395c-45e5-ad57-a570f44d9fa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '347d9d75c5064f9fa5b7c6528d464688', 'request_id': 'req-c67f5aec-395c-45e5-ad57-a570f44d9fa0'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:18.884708 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c67f5aec-395c-45e5-ad57-a570f44d9fa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '347d9d75c5064f9fa5b7c6528d464688', 'request_id': 'req-c67f5aec-395c-45e5-ad57-a570f44d9fa0'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:18.893522 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c67f5aec-395c-45e5-ad57-a570f44d9fa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '347d9d75c5064f9fa5b7c6528d464688', 'request_id': 'req-c67f5aec-395c-45e5-ad57-a570f44d9fa0'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:18.915039 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:18.923456 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c67f5aec-395c-45e5-ad57-a570f44d9fa0 admin admin] The token cache is being invalidate because project 347d9d75c5064f9fa5b7c6528d464688 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:40:18.925007 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c67f5aec-395c-45e5-ad57-a570f44d9fa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:18.926906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c67f5aec-395c-45e5-ad57-a570f44d9fa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:18.930939 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1288/2558] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:40:18 2022] DELETE /identity/v3/projects/347d9d75c5064f9fa5b7c6528d464688 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:40:18.936053 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f5aec-395c-45e5-ad57-a570f44d9fa0 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:18.942531 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c67f5aec-395c-45e5-ad57-a570f44d9fa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:18.944986 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3e3bf-f059-4eaf-8466-92369725254c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:18.945766 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3e3bf-f059-4eaf-8466-92369725254c admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:18.946134 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3e3bf-f059-4eaf-8466-92369725254c admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:18.946479 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3e3bf-f059-4eaf-8466-92369725254c admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:18.947265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3e3bf-f059-4eaf-8466-92369725254c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:18.949388 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3e3bf-f059-4eaf-8466-92369725254c admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:18.966482 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67f5aec-395c-45e5-ad57-a570f44d9fa0 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:18.971843 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e430ae1-1e6c-46a1-b019-b7f986ab6b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:18.973189 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e430ae1-1e6c-46a1-b019-b7f986ab6b8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:18.973189 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e430ae1-1e6c-46a1-b019-b7f986ab6b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:18.973528 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e430ae1-1e6c-46a1-b019-b7f986ab6b8e admin admin] PATH_INFO: `/v3/users/3e71c079db234c1db9366f57121c62b4` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:18.974739 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e430ae1-1e6c-46a1-b019-b7f986ab6b8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e71c079db234c1db9366f57121c62b4)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:18.977030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e430ae1-1e6c-46a1-b019-b7f986ab6b8e admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:19.008928 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1271/2559] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:40:18 2022] POST /identity/v3/projects => generated 385 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:40:19.013453 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3e3bf-f059-4eaf-8466-92369725254c admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:19.042481 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-6be3e3bf-f059-4eaf-8466-92369725254c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:19.075356 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8e430ae1-1e6c-46a1-b019-b7f986ab6b8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e71c079db234c1db9366f57121c62b4'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:19.090839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8e430ae1-1e6c-46a1-b019-b7f986ab6b8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e71c079db234c1db9366f57121c62b4'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:19.118578 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:19.121451 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8e430ae1-1e6c-46a1-b019-b7f986ab6b8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e71c079db234c1db9366f57121c62b4'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:19.122120 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8e430ae1-1e6c-46a1-b019-b7f986ab6b8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e71c079db234c1db9366f57121c62b4'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:19.122883 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8e430ae1-1e6c-46a1-b019-b7f986ab6b8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e71c079db234c1db9366f57121c62b4'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:19.146397 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1289/2560] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:40:18 2022] DELETE /identity/v3/users/3e71c079db234c1db9366f57121c62b4 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:40:19.182125 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3e3bf-f059-4eaf-8466-92369725254c admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:19.296888 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dac5be-6866-4de9-beb0-570e6e2873be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:19.297657 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dac5be-6866-4de9-beb0-570e6e2873be admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:19.298057 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dac5be-6866-4de9-beb0-570e6e2873be admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:19.298341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dac5be-6866-4de9-beb0-570e6e2873be admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:19.299042 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dac5be-6866-4de9-beb0-570e6e2873be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:19.302076 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dac5be-6866-4de9-beb0-570e6e2873be admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:19.333843 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1272/2561] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:40:19 2022] POST /identity/v3/users => generated 299 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:40:19.335052 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e430ae1-1e6c-46a1-b019-b7f986ab6b8e admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:19.338023 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-8e430ae1-1e6c-46a1-b019-b7f986ab6b8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:19.355460 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e430ae1-1e6c-46a1-b019-b7f986ab6b8e admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:19.365347 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b092e20-3b61-4142-9bc2-c360f7c5eeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:19.366569 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b092e20-3b61-4142-9bc2-c360f7c5eeeb admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:19.367082 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b092e20-3b61-4142-9bc2-c360f7c5eeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:19.367699 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b092e20-3b61-4142-9bc2-c360f7c5eeeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:19.369352 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b092e20-3b61-4142-9bc2-c360f7c5eeeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:19.371193 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b092e20-3b61-4142-9bc2-c360f7c5eeeb admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:19.389087 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dac5be-6866-4de9-beb0-570e6e2873be admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:19.390159 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1290/2562] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:40:19.413417 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dac5be-6866-4de9-beb0-570e6e2873be admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:19.429336 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289dc27-05cf-4996-9c4d-b0a6ff487484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:19.430481 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289dc27-05cf-4996-9c4d-b0a6ff487484 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:19.431007 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289dc27-05cf-4996-9c4d-b0a6ff487484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:19.431392 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f289dc27-05cf-4996-9c4d-b0a6ff487484 admin admin] PATH_INFO: `/v3/projects/332fb4ca06ce4371b401826a87fbe3cc/users/764ca3d7ba814eba89b650c76f53ec57/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:19.460547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f289dc27-05cf-4996-9c4d-b0a6ff487484 admin admin] RBAC: Authorizing `identity:create_grant(project_id=332fb4ca06ce4371b401826a87fbe3cc, user_id=764ca3d7ba814eba89b650c76f53ec57, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:19.463776 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f289dc27-05cf-4996-9c4d-b0a6ff487484 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:19.487402 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1273/2563] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:40:19 2022] PUT /identity/v3/projects/332fb4ca06ce4371b401826a87fbe3cc/users/764ca3d7ba814eba89b650c76f53ec57/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:19.500292 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b092e20-3b61-4142-9bc2-c360f7c5eeeb admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:19.507669 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b092e20-3b61-4142-9bc2-c360f7c5eeeb admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:19.616679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd770bed-b7aa-49e0-ad49-aa12a4fc0e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:19.617794 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd770bed-b7aa-49e0-ad49-aa12a4fc0e1a admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:19.618293 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd770bed-b7aa-49e0-ad49-aa12a4fc0e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:19.618829 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd770bed-b7aa-49e0-ad49-aa12a4fc0e1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:19.620760 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd770bed-b7aa-49e0-ad49-aa12a4fc0e1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:19.635676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd770bed-b7aa-49e0-ad49-aa12a4fc0e1a admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:19.650874 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289dc27-05cf-4996-9c4d-b0a6ff487484 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:19.653183 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-f289dc27-05cf-4996-9c4d-b0a6ff487484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:19.656429 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1291/2564] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:40:19 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:40:19.661249 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd770bed-b7aa-49e0-ad49-aa12a4fc0e1a admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:19.685778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f289dc27-05cf-4996-9c4d-b0a6ff487484 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:19.694981 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc21c05-e554-4f38-aa39-418c6cab52da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:19.695980 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc21c05-e554-4f38-aa39-418c6cab52da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:19.696632 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc21c05-e554-4f38-aa39-418c6cab52da admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:19.697014 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc21c05-e554-4f38-aa39-418c6cab52da admin admin] PATH_INFO: `/v3/projects/9124993073634c6eaa41a90c0710f12a` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:19.697909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd770bed-b7aa-49e0-ad49-aa12a4fc0e1a admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:19.698628 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc21c05-e554-4f38-aa39-418c6cab52da admin admin] RBAC: Authorizing `identity:delete_project(project_id=9124993073634c6eaa41a90c0710f12a)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:19.700942 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc21c05-e554-4f38-aa39-418c6cab52da admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:19.706358 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffdc6bc-3ad6-4d39-b86e-adcf81a1fce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:19.707435 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffdc6bc-3ad6-4d39-b86e-adcf81a1fce3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:19.707794 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffdc6bc-3ad6-4d39-b86e-adcf81a1fce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:19.708201 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffdc6bc-3ad6-4d39-b86e-adcf81a1fce3 admin admin] PATH_INFO: `/v3/projects/332fb4ca06ce4371b401826a87fbe3cc/users/764ca3d7ba814eba89b650c76f53ec57/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:19.711038 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffdc6bc-3ad6-4d39-b86e-adcf81a1fce3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=332fb4ca06ce4371b401826a87fbe3cc, user_id=764ca3d7ba814eba89b650c76f53ec57, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:19.715021 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffdc6bc-3ad6-4d39-b86e-adcf81a1fce3 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:19.733765 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1292/2565] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:40:19 2022] PUT /identity/v3/projects/332fb4ca06ce4371b401826a87fbe3cc/users/764ca3d7ba814eba89b650c76f53ec57/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:40:19.745745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b9be7-af98-4be3-a3d2-cfcd1528a028 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:19.746029 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b9be7-af98-4be3-a3d2-cfcd1528a028 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:19.746324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b9be7-af98-4be3-a3d2-cfcd1528a028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:19.783235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-023b9be7-af98-4be3-a3d2-cfcd1528a028 None None] MFA Rules not processed for user `764ca3d7ba814eba89b650c76f53ec57`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:19.785650 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-023b9be7-af98-4be3-a3d2-cfcd1528a028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:19.809754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-8dc21c05-e554-4f38-aa39-418c6cab52da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9124993073634c6eaa41a90c0710f12a', 'request_id': 'req-8dc21c05-e554-4f38-aa39-418c6cab52da'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:19.820529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-8dc21c05-e554-4f38-aa39-418c6cab52da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9124993073634c6eaa41a90c0710f12a', 'request_id': 'req-8dc21c05-e554-4f38-aa39-418c6cab52da'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:19.835677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-8dc21c05-e554-4f38-aa39-418c6cab52da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9124993073634c6eaa41a90c0710f12a', 'request_id': 'req-8dc21c05-e554-4f38-aa39-418c6cab52da'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:19.845941 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:19.853653 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-8dc21c05-e554-4f38-aa39-418c6cab52da admin admin] The token cache is being invalidate because project 9124993073634c6eaa41a90c0710f12a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:40:19.853989 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-8dc21c05-e554-4f38-aa39-418c6cab52da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:19.855012 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-8dc21c05-e554-4f38-aa39-418c6cab52da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:19.860213 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1274/2566] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:40:19 2022] DELETE /identity/v3/projects/9124993073634c6eaa41a90c0710f12a => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:19.863940 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5cbca-bb3c-467f-b89e-884c096db7b5 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:19.864403 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5cbca-bb3c-467f-b89e-884c096db7b5 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:19.865162 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5cbca-bb3c-467f-b89e-884c096db7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:19.892083 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-88e5cbca-bb3c-467f-b89e-884c096db7b5 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:19.893753 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-88e5cbca-bb3c-467f-b89e-884c096db7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:19.931329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:19.985375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1293/2567] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:19 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:19.997320 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b9be7-af98-4be3-a3d2-cfcd1528a028 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:20.003666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-023b9be7-af98-4be3-a3d2-cfcd1528a028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:20.014932 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1275/2568] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:40:20.049713 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7c2d8-54ee-406b-b0b8-dc6e9d8a0ef9 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:20.050225 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7c2d8-54ee-406b-b0b8-dc6e9d8a0ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:20.051610 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7c2d8-54ee-406b-b0b8-dc6e9d8a0ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:20.114164 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-adf7c2d8-54ee-406b-b0b8-dc6e9d8a0ef9 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:20.116072 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-adf7c2d8-54ee-406b-b0b8-dc6e9d8a0ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:20.143747 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1276/2569] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:40:20.150522 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7c2d8-54ee-406b-b0b8-dc6e9d8a0ef9 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:20.169695 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7c2d8-54ee-406b-b0b8-dc6e9d8a0ef9 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:20.176123 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ab0fc-e102-406e-84a6-39703ef1962d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:20.176881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ab0fc-e102-406e-84a6-39703ef1962d admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:20.177177 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ab0fc-e102-406e-84a6-39703ef1962d admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:20.177463 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594ab0fc-e102-406e-84a6-39703ef1962d admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:20.178213 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ab0fc-e102-406e-84a6-39703ef1962d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:40:20.178528 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ab0fc-e102-406e-84a6-39703ef1962d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:20.181333 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594ab0fc-e102-406e-84a6-39703ef1962d admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:20.231775 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1277/2570] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:40:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:40:20.241040 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ab0fc-e102-406e-84a6-39703ef1962d admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:20.256624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b9be7-af98-4be3-a3d2-cfcd1528a028 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:20.299888 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594ab0fc-e102-406e-84a6-39703ef1962d admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:20.303233 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67830d73-1a5e-4588-a1bc-2f2d75c2e268 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:20.304321 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67830d73-1a5e-4588-a1bc-2f2d75c2e268 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:20.304781 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67830d73-1a5e-4588-a1bc-2f2d75c2e268 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:20.305257 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67830d73-1a5e-4588-a1bc-2f2d75c2e268 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:20.325771 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003ab3f-1059-4130-abcb-c1f7d5702611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:20.327016 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003ab3f-1059-4130-abcb-c1f7d5702611 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:20.327497 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003ab3f-1059-4130-abcb-c1f7d5702611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:20.327970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003ab3f-1059-4130-abcb-c1f7d5702611 admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:20.329504 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8003ab3f-1059-4130-abcb-c1f7d5702611 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:20.333087 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8003ab3f-1059-4130-abcb-c1f7d5702611 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:20.339014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67830d73-1a5e-4588-a1bc-2f2d75c2e268 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:20.341631 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67830d73-1a5e-4588-a1bc-2f2d75c2e268 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:20.378005 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1294/2571] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:40:19 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:40:20.445063 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1278/2572] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:40:20 2022] POST /identity/v3/projects => generated 381 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:40:20.450026 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67830d73-1a5e-4588-a1bc-2f2d75c2e268 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:20.480948 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67830d73-1a5e-4588-a1bc-2f2d75c2e268 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:20.606204 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d68665-70c6-4dea-b0c6-489f40ddf1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:20.607238 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d68665-70c6-4dea-b0c6-489f40ddf1fd admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:20.607741 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d68665-70c6-4dea-b0c6-489f40ddf1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:20.609420 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d68665-70c6-4dea-b0c6-489f40ddf1fd admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:20.609627 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d68665-70c6-4dea-b0c6-489f40ddf1fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:20.612492 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d68665-70c6-4dea-b0c6-489f40ddf1fd admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:20.667012 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003ab3f-1059-4130-abcb-c1f7d5702611 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:20.667461 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1295/2573] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:40:20 2022] POST /identity/v3/users => generated 291 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:40:20.685265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003ab3f-1059-4130-abcb-c1f7d5702611 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:20.695528 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d23b9-c0dd-41b5-868b-91c12878c92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:20.697093 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d23b9-c0dd-41b5-868b-91c12878c92e admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:20.697579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d23b9-c0dd-41b5-868b-91c12878c92e admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:20.698170 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d23b9-c0dd-41b5-868b-91c12878c92e admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:20.699752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d23b9-c0dd-41b5-868b-91c12878c92e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:20.702842 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d23b9-c0dd-41b5-868b-91c12878c92e admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:20.732775 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d68665-70c6-4dea-b0c6-489f40ddf1fd admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:20.742710 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1279/2574] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:40:20.755785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d68665-70c6-4dea-b0c6-489f40ddf1fd admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:20.771836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043a465-6329-467f-ae34-9fd52f268268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:20.772582 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043a465-6329-467f-ae34-9fd52f268268 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:20.773435 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043a465-6329-467f-ae34-9fd52f268268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:20.773918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043a465-6329-467f-ae34-9fd52f268268 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:20.779372 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a043a465-6329-467f-ae34-9fd52f268268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:20.782925 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a043a465-6329-467f-ae34-9fd52f268268 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:20.822378 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1296/2575] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:40:20 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:40:20.828685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d23b9-c0dd-41b5-868b-91c12878c92e admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:20.869171 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d23b9-c0dd-41b5-868b-91c12878c92e admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:20.880203 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20060499-d52b-438b-9abc-7bc4542dc86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:20.881953 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20060499-d52b-438b-9abc-7bc4542dc86c admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:20.884531 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20060499-d52b-438b-9abc-7bc4542dc86c admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:20.885124 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20060499-d52b-438b-9abc-7bc4542dc86c admin admin] PATH_INFO: `/v3/projects/8df9f2d2770741e8ba3497f769680b91/users/79db7ba8e1fe42dbb4a98f6191d483f3/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:20.943312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20060499-d52b-438b-9abc-7bc4542dc86c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8df9f2d2770741e8ba3497f769680b91, user_id=79db7ba8e1fe42dbb4a98f6191d483f3, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:20.958985 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20060499-d52b-438b-9abc-7bc4542dc86c admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:21.003005 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1280/2576] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:40:20 2022] PUT /identity/v3/projects/8df9f2d2770741e8ba3497f769680b91/users/79db7ba8e1fe42dbb4a98f6191d483f3/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:21.023238 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf1981-8bba-4dfd-b803-d40e57921586 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:21.024342 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf1981-8bba-4dfd-b803-d40e57921586 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:21.024852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf1981-8bba-4dfd-b803-d40e57921586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:21.088030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-9adf1981-8bba-4dfd-b803-d40e57921586 None None] MFA Rules not processed for user `79db7ba8e1fe42dbb4a98f6191d483f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:21.090149 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-9adf1981-8bba-4dfd-b803-d40e57921586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:21.202968 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:21.264203 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1297/2577] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:21.271921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20060499-d52b-438b-9abc-7bc4542dc86c admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:21.463706 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20060499-d52b-438b-9abc-7bc4542dc86c admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:21.476361 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be70b822-7a4c-4a0a-8e09-69414083289f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:21.477800 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70b822-7a4c-4a0a-8e09-69414083289f admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:21.478346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70b822-7a4c-4a0a-8e09-69414083289f admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:21.485995 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70b822-7a4c-4a0a-8e09-69414083289f admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:21.487251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be70b822-7a4c-4a0a-8e09-69414083289f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:21.489739 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be70b822-7a4c-4a0a-8e09-69414083289f admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:21.554071 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1281/2578] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:40:21 2022] POST /identity/v3/projects => generated 381 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:40:21.561420 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf1981-8bba-4dfd-b803-d40e57921586 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:21.600733 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf1981-8bba-4dfd-b803-d40e57921586 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:21.680869 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be70b822-7a4c-4a0a-8e09-69414083289f admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:21.682993 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-be70b822-7a4c-4a0a-8e09-69414083289f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:21.740791 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53467a85-032e-4991-aadf-16583fab3254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:21.741487 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53467a85-032e-4991-aadf-16583fab3254 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:21.741775 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53467a85-032e-4991-aadf-16583fab3254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:21.742066 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53467a85-032e-4991-aadf-16583fab3254 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:21.742787 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53467a85-032e-4991-aadf-16583fab3254 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:21.745457 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53467a85-032e-4991-aadf-16583fab3254 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:21.784162 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1298/2579] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:40:21 2022] POST /identity/v3/users => generated 297 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:40:21.784425 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:21.786236 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53467a85-032e-4991-aadf-16583fab3254 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:21.807037 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53467a85-032e-4991-aadf-16583fab3254 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:21.820492 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f710d95-de26-4803-944f-68a3c617f927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:21.821190 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f710d95-de26-4803-944f-68a3c617f927 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:21.821464 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f710d95-de26-4803-944f-68a3c617f927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:21.821745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f710d95-de26-4803-944f-68a3c617f927 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:21.822693 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f710d95-de26-4803-944f-68a3c617f927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:21.824345 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f710d95-de26-4803-944f-68a3c617f927 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:21.825011 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be70b822-7a4c-4a0a-8e09-69414083289f admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:21.836209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca50904-89ee-488c-b0f7-f9f05b253fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:21.836209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca50904-89ee-488c-b0f7-f9f05b253fc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:21.836209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca50904-89ee-488c-b0f7-f9f05b253fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:21.836209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca50904-89ee-488c-b0f7-f9f05b253fc5 admin admin] PATH_INFO: `/v3/users/f586f39e308847aebced04e3d149656c` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:21.836868 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca50904-89ee-488c-b0f7-f9f05b253fc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f586f39e308847aebced04e3d149656c)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:21.842717 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca50904-89ee-488c-b0f7-f9f05b253fc5 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:21.851138 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1299/2580] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:40:21.853020 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f710d95-de26-4803-944f-68a3c617f927 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:21.880657 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f710d95-de26-4803-944f-68a3c617f927 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:21.889899 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0463a4e-269d-4316-ba45-1d3fbf4cedbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:21.890955 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0463a4e-269d-4316-ba45-1d3fbf4cedbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:21.891281 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0463a4e-269d-4316-ba45-1d3fbf4cedbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:21.891657 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0463a4e-269d-4316-ba45-1d3fbf4cedbf admin admin] PATH_INFO: `/v3/projects/d5ae2b406d8b4a80a0de19c7d7f4c2df/users/b264dcde78a34dcb827a9d4e35c15a41/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:21.924027 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0463a4e-269d-4316-ba45-1d3fbf4cedbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5ae2b406d8b4a80a0de19c7d7f4c2df, user_id=b264dcde78a34dcb827a9d4e35c15a41, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:21.930293 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0463a4e-269d-4316-ba45-1d3fbf4cedbf admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:21.946108 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bca50904-89ee-488c-b0f7-f9f05b253fc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f586f39e308847aebced04e3d149656c'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:21.952700 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bca50904-89ee-488c-b0f7-f9f05b253fc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f586f39e308847aebced04e3d149656c'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:21.953282 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1300/2581] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:40:21 2022] PUT /identity/v3/projects/d5ae2b406d8b4a80a0de19c7d7f4c2df/users/b264dcde78a34dcb827a9d4e35c15a41/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:40:21.963765 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bca50904-89ee-488c-b0f7-f9f05b253fc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f586f39e308847aebced04e3d149656c'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:21.964886 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0463a4e-269d-4316-ba45-1d3fbf4cedbf admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:21.981976 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bca50904-89ee-488c-b0f7-f9f05b253fc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f586f39e308847aebced04e3d149656c'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:22.006569 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bca50904-89ee-488c-b0f7-f9f05b253fc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f586f39e308847aebced04e3d149656c'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:22.034293 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1282/2582] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:40:21 2022] DELETE /identity/v3/users/f586f39e308847aebced04e3d149656c => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:40:22.037251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8fdad-0bf0-4562-9d7e-8fc1e40369f9 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:22.037748 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8fdad-0bf0-4562-9d7e-8fc1e40369f9 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:22.038509 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e8fdad-0bf0-4562-9d7e-8fc1e40369f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:22.060458 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-c5e8fdad-0bf0-4562-9d7e-8fc1e40369f9 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:22.061874 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-c5e8fdad-0bf0-4562-9d7e-8fc1e40369f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:22.151509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0463a4e-269d-4316-ba45-1d3fbf4cedbf admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:22.159476 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd568a49-116f-4c2a-985f-a987b114dc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:22.160987 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd568a49-116f-4c2a-985f-a987b114dc79 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:22.161656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd568a49-116f-4c2a-985f-a987b114dc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:22.162233 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd568a49-116f-4c2a-985f-a987b114dc79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:22.163706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd568a49-116f-4c2a-985f-a987b114dc79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:22.167120 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd568a49-116f-4c2a-985f-a987b114dc79 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:22.169404 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1283/2583] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:22.178038 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8fdad-0bf0-4562-9d7e-8fc1e40369f9 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:22.181294 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-c5e8fdad-0bf0-4562-9d7e-8fc1e40369f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:22.199348 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1301/2584] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:40:22.204145 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd568a49-116f-4c2a-985f-a987b114dc79 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:22.207849 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-cd568a49-116f-4c2a-985f-a987b114dc79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:22.239948 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd568a49-116f-4c2a-985f-a987b114dc79 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:22.246043 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bb985-fbc0-43d9-8137-8c6604077ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:22.247724 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bb985-fbc0-43d9-8137-8c6604077ec3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:22.248122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bb985-fbc0-43d9-8137-8c6604077ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:22.248652 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bb985-fbc0-43d9-8137-8c6604077ec3 admin admin] PATH_INFO: `/v3/projects/d5ae2b406d8b4a80a0de19c7d7f4c2df/users/b264dcde78a34dcb827a9d4e35c15a41/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:22.251437 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6bb985-fbc0-43d9-8137-8c6604077ec3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5ae2b406d8b4a80a0de19c7d7f4c2df, user_id=b264dcde78a34dcb827a9d4e35c15a41, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:22.255471 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6bb985-fbc0-43d9-8137-8c6604077ec3 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:22.276301 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1302/2585] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:40:22 2022] PUT /identity/v3/projects/d5ae2b406d8b4a80a0de19c7d7f4c2df/users/b264dcde78a34dcb827a9d4e35c15a41/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:40:22.295221 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f344d-6636-4fa2-9540-e7749b3d3709 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:22.295574 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f344d-6636-4fa2-9540-e7749b3d3709 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:22.296039 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f344d-6636-4fa2-9540-e7749b3d3709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:22.297760 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e8fdad-0bf0-4562-9d7e-8fc1e40369f9 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:22.316852 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:22.353338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-0a5f344d-6636-4fa2-9540-e7749b3d3709 None None] MFA Rules not processed for user `b264dcde78a34dcb827a9d4e35c15a41`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:22.362717 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-0a5f344d-6636-4fa2-9540-e7749b3d3709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:22.369457 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:22.430793 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de9ab3-247d-4e9e-8f83-a77b7121247a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:22.431907 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de9ab3-247d-4e9e-8f83-a77b7121247a service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:22.434272 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de9ab3-247d-4e9e-8f83-a77b7121247a service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:22.434272 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de9ab3-247d-4e9e-8f83-a77b7121247a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:22.583454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1303/2586] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:22 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:40:22.588423 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f344d-6636-4fa2-9540-e7749b3d3709 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:22.588423 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-0a5f344d-6636-4fa2-9540-e7749b3d3709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:22.628002 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2de9ab3-247d-4e9e-8f83-a77b7121247a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:22.630646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2de9ab3-247d-4e9e-8f83-a77b7121247a service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:22.651123 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1284/2587] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:40:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:40:22.748917 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f344d-6636-4fa2-9540-e7749b3d3709 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:22.760591 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e9dca-03a6-4e2a-9a32-9bb630fb4a00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:22.761679 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e9dca-03a6-4e2a-9a32-9bb630fb4a00 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:22.762330 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e9dca-03a6-4e2a-9a32-9bb630fb4a00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:22.762760 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e9dca-03a6-4e2a-9a32-9bb630fb4a00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:22.793693 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e9dca-03a6-4e2a-9a32-9bb630fb4a00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:22.797364 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e9dca-03a6-4e2a-9a32-9bb630fb4a00 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:22.818066 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1304/2588] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:40:22 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:40:23.014671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de9ab3-247d-4e9e-8f83-a77b7121247a service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:23.041125 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de9ab3-247d-4e9e-8f83-a77b7121247a service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:23.045807 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f1d660-b4ae-423f-831e-66fce28cc1af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:23.046581 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1d660-b4ae-423f-831e-66fce28cc1af service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:23.047451 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1d660-b4ae-423f-831e-66fce28cc1af service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:23.047867 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1d660-b4ae-423f-831e-66fce28cc1af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:23.051538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-d7f1d660-b4ae-423f-831e-66fce28cc1af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:23.160699 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f1d660-b4ae-423f-831e-66fce28cc1af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:23.162093 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f1d660-b4ae-423f-831e-66fce28cc1af service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:23.172006 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1285/2589] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:40:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:40:23.290783 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e9dca-03a6-4e2a-9a32-9bb630fb4a00 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:23.293248 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e18e9dca-03a6-4e2a-9a32-9bb630fb4a00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:23.336779 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e9dca-03a6-4e2a-9a32-9bb630fb4a00 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:23.341567 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c31e93-0698-4ee9-b05b-22f21ae7cbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:23.342442 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c31e93-0698-4ee9-b05b-22f21ae7cbcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:23.343004 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c31e93-0698-4ee9-b05b-22f21ae7cbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:23.343251 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c31e93-0698-4ee9-b05b-22f21ae7cbcb admin admin] PATH_INFO: `/v3/projects/c7193dfd80704d9ab9b4c0ded7c938d1` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:23.344493 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c31e93-0698-4ee9-b05b-22f21ae7cbcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7193dfd80704d9ab9b4c0ded7c938d1)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:23.346425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c31e93-0698-4ee9-b05b-22f21ae7cbcb admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:23.408830 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-04c31e93-0698-4ee9-b05b-22f21ae7cbcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7193dfd80704d9ab9b4c0ded7c938d1', 'request_id': 'req-04c31e93-0698-4ee9-b05b-22f21ae7cbcb'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:23.415420 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-04c31e93-0698-4ee9-b05b-22f21ae7cbcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7193dfd80704d9ab9b4c0ded7c938d1', 'request_id': 'req-04c31e93-0698-4ee9-b05b-22f21ae7cbcb'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:23.422586 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-04c31e93-0698-4ee9-b05b-22f21ae7cbcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7193dfd80704d9ab9b4c0ded7c938d1', 'request_id': 'req-04c31e93-0698-4ee9-b05b-22f21ae7cbcb'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:23.437387 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-04c31e93-0698-4ee9-b05b-22f21ae7cbcb admin admin] The token cache is being invalidate because project c7193dfd80704d9ab9b4c0ded7c938d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:40:23.437681 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-04c31e93-0698-4ee9-b05b-22f21ae7cbcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:23.438345 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-04c31e93-0698-4ee9-b05b-22f21ae7cbcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:23.439942 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1305/2590] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:40:23 2022] DELETE /identity/v3/projects/c7193dfd80704d9ab9b4c0ded7c938d1 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:23.445522 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f1d660-b4ae-423f-831e-66fce28cc1af service cinder] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:23.460759 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f1d660-b4ae-423f-831e-66fce28cc1af service cinder] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:23.550655 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96814cce-69e0-4bd1-a6d8-a7a7f8f4195b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:23.552194 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96814cce-69e0-4bd1-a6d8-a7a7f8f4195b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:23.552489 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96814cce-69e0-4bd1-a6d8-a7a7f8f4195b admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:23.552769 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96814cce-69e0-4bd1-a6d8-a7a7f8f4195b admin admin] PATH_INFO: `/v3/users/764ca3d7ba814eba89b650c76f53ec57` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:23.554027 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96814cce-69e0-4bd1-a6d8-a7a7f8f4195b admin admin] RBAC: Authorizing `identity:delete_user(user_id=764ca3d7ba814eba89b650c76f53ec57)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:23.556033 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96814cce-69e0-4bd1-a6d8-a7a7f8f4195b admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:23.558151 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:23.615537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-96814cce-69e0-4bd1-a6d8-a7a7f8f4195b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '764ca3d7ba814eba89b650c76f53ec57'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:23.616249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-96814cce-69e0-4bd1-a6d8-a7a7f8f4195b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '764ca3d7ba814eba89b650c76f53ec57'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:23.623832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-96814cce-69e0-4bd1-a6d8-a7a7f8f4195b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '764ca3d7ba814eba89b650c76f53ec57'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:23.639280 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-96814cce-69e0-4bd1-a6d8-a7a7f8f4195b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '764ca3d7ba814eba89b650c76f53ec57'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:23.647144 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-96814cce-69e0-4bd1-a6d8-a7a7f8f4195b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '764ca3d7ba814eba89b650c76f53ec57'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:23.653651 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1286/2591] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:40:23 2022] DELETE /identity/v3/users/764ca3d7ba814eba89b650c76f53ec57 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:23.950406 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c31e93-0698-4ee9-b05b-22f21ae7cbcb admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:23.952824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-04c31e93-0698-4ee9-b05b-22f21ae7cbcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:23.974389 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c31e93-0698-4ee9-b05b-22f21ae7cbcb admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:24.015193 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:24.059784 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4591329d-8d67-49a0-b6b2-3b69d9cbcd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:24.061249 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591329d-8d67-49a0-b6b2-3b69d9cbcd9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:24.061752 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591329d-8d67-49a0-b6b2-3b69d9cbcd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:24.062146 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4591329d-8d67-49a0-b6b2-3b69d9cbcd9d admin admin] PATH_INFO: `/v3/projects/332fb4ca06ce4371b401826a87fbe3cc` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:24.064385 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591329d-8d67-49a0-b6b2-3b69d9cbcd9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=332fb4ca06ce4371b401826a87fbe3cc)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:24.069192 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4591329d-8d67-49a0-b6b2-3b69d9cbcd9d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:24.126513 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4591329d-8d67-49a0-b6b2-3b69d9cbcd9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '332fb4ca06ce4371b401826a87fbe3cc', 'request_id': 'req-4591329d-8d67-49a0-b6b2-3b69d9cbcd9d'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:24.133058 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4591329d-8d67-49a0-b6b2-3b69d9cbcd9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '332fb4ca06ce4371b401826a87fbe3cc', 'request_id': 'req-4591329d-8d67-49a0-b6b2-3b69d9cbcd9d'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:24.139235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4591329d-8d67-49a0-b6b2-3b69d9cbcd9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '332fb4ca06ce4371b401826a87fbe3cc', 'request_id': 'req-4591329d-8d67-49a0-b6b2-3b69d9cbcd9d'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:24.154352 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4591329d-8d67-49a0-b6b2-3b69d9cbcd9d admin admin] The token cache is being invalidate because project 332fb4ca06ce4371b401826a87fbe3cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:40:24.154853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4591329d-8d67-49a0-b6b2-3b69d9cbcd9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:24.155823 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-4591329d-8d67-49a0-b6b2-3b69d9cbcd9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:24.158270 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1306/2592] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:40:23 2022] DELETE /identity/v3/projects/332fb4ca06ce4371b401826a87fbe3cc => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:24.166117 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc155da1-67af-4592-9c5e-6cb2b4b0a061 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:24.167382 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc155da1-67af-4592-9c5e-6cb2b4b0a061 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:24.168116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc155da1-67af-4592-9c5e-6cb2b4b0a061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:24.198063 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-fc155da1-67af-4592-9c5e-6cb2b4b0a061 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:24.200066 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-fc155da1-67af-4592-9c5e-6cb2b4b0a061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:24.281840 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1287/2593] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:24.290297 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c1564-e52e-4f40-ac65-61c87d914cec None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:24.290580 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c1564-e52e-4f40-ac65-61c87d914cec None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:24.290860 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c1564-e52e-4f40-ac65-61c87d914cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:24.314333 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-196c1564-e52e-4f40-ac65-61c87d914cec None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:24.315446 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-196c1564-e52e-4f40-ac65-61c87d914cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:24.324819 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1307/2594] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:40:24.329326 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc155da1-67af-4592-9c5e-6cb2b4b0a061 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:24.340504 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc155da1-67af-4592-9c5e-6cb2b4b0a061 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:24.348102 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ad6ab-7e13-410a-88a8-83d724634b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:24.349274 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ad6ab-7e13-410a-88a8-83d724634b8b admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:24.350188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ad6ab-7e13-410a-88a8-83d724634b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:24.350188 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ad6ab-7e13-410a-88a8-83d724634b8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:24.351570 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ad6ab-7e13-410a-88a8-83d724634b8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:40:24.352129 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ad6ab-7e13-410a-88a8-83d724634b8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:24.355287 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ad6ab-7e13-410a-88a8-83d724634b8b admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:24.378037 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1288/2595] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:40:24.383022 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c1564-e52e-4f40-ac65-61c87d914cec None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:24.394965 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c1564-e52e-4f40-ac65-61c87d914cec None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:24.400054 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f282d1-6663-41f4-a7ed-8b7e6073d158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:24.400880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f282d1-6663-41f4-a7ed-8b7e6073d158 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:24.401104 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f282d1-6663-41f4-a7ed-8b7e6073d158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:24.401379 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f282d1-6663-41f4-a7ed-8b7e6073d158 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:24.402073 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f282d1-6663-41f4-a7ed-8b7e6073d158 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:24.403995 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f282d1-6663-41f4-a7ed-8b7e6073d158 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:24.435941 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1308/2596] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:40:24 2022] POST /identity/v3/projects => generated 361 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:40:24.438452 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ad6ab-7e13-410a-88a8-83d724634b8b admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:24.448784 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ad6ab-7e13-410a-88a8-83d724634b8b admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:24.506781 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0efceae-0f56-457b-b5e5-5e83a354c869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:24.507454 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0efceae-0f56-457b-b5e5-5e83a354c869 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:24.507739 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0efceae-0f56-457b-b5e5-5e83a354c869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:24.508015 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0efceae-0f56-457b-b5e5-5e83a354c869 admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:24.508760 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0efceae-0f56-457b-b5e5-5e83a354c869 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:24.510688 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0efceae-0f56-457b-b5e5-5e83a354c869 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:24.521265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:24.532201 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1289/2597] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:40:24 2022] POST /identity/v3/users => generated 281 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:40:24.536591 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f282d1-6663-41f4-a7ed-8b7e6073d158 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:24.546853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f282d1-6663-41f4-a7ed-8b7e6073d158 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:24.551802 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87126c-1fa3-4050-8627-6d2982b57897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:24.552516 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87126c-1fa3-4050-8627-6d2982b57897 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:24.552800 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87126c-1fa3-4050-8627-6d2982b57897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:24.553074 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87126c-1fa3-4050-8627-6d2982b57897 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:24.553839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b87126c-1fa3-4050-8627-6d2982b57897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:24.555396 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b87126c-1fa3-4050-8627-6d2982b57897 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:24.567088 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1309/2598] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:40:24.571393 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0efceae-0f56-457b-b5e5-5e83a354c869 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:24.582846 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0efceae-0f56-457b-b5e5-5e83a354c869 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:24.587867 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817ad91-3b3e-4dc6-92a9-8cc61164e144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:24.588614 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817ad91-3b3e-4dc6-92a9-8cc61164e144 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:24.588900 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817ad91-3b3e-4dc6-92a9-8cc61164e144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:24.589178 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817ad91-3b3e-4dc6-92a9-8cc61164e144 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:24.589904 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a817ad91-3b3e-4dc6-92a9-8cc61164e144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:24.591492 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a817ad91-3b3e-4dc6-92a9-8cc61164e144 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:24.609736 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1290/2599] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:40:24.610919 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87126c-1fa3-4050-8627-6d2982b57897 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:24.626846 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87126c-1fa3-4050-8627-6d2982b57897 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:24.633302 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ce933-e860-4190-81bc-cb80fe49715e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:24.634205 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ce933-e860-4190-81bc-cb80fe49715e admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:24.634480 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ce933-e860-4190-81bc-cb80fe49715e admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:24.634886 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ce933-e860-4190-81bc-cb80fe49715e admin admin] PATH_INFO: `/v3/projects/b1c77009657c41fd9922e8828cfd7f38/users/9e6d3d00e2a24f84974d28da937779ff/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:24.652023 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ce933-e860-4190-81bc-cb80fe49715e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1c77009657c41fd9922e8828cfd7f38, user_id=9e6d3d00e2a24f84974d28da937779ff, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:24.654777 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ce933-e860-4190-81bc-cb80fe49715e admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:24.670660 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1310/2600] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:40:24 2022] PUT /identity/v3/projects/b1c77009657c41fd9922e8828cfd7f38/users/9e6d3d00e2a24f84974d28da937779ff/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:24.684357 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e989a-4a23-474b-a68a-7c50999b3416 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:24.684662 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e989a-4a23-474b-a68a-7c50999b3416 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:24.684954 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e989a-4a23-474b-a68a-7c50999b3416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:24.718233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-e98e989a-4a23-474b-a68a-7c50999b3416 None None] MFA Rules not processed for user `9e6d3d00e2a24f84974d28da937779ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:24.720150 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-e98e989a-4a23-474b-a68a-7c50999b3416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:24.807723 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1291/2601] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:24 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:40:24.844647 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ce933-e860-4190-81bc-cb80fe49715e admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:24.846365 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e67ce933-e860-4190-81bc-cb80fe49715e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:24.908039 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ce933-e860-4190-81bc-cb80fe49715e admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:24.912651 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8603ec9-0cbf-4dc3-a0ee-5a765f1e3fe1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:24.913370 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8603ec9-0cbf-4dc3-a0ee-5a765f1e3fe1 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:24.913657 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8603ec9-0cbf-4dc3-a0ee-5a765f1e3fe1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:24.913930 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8603ec9-0cbf-4dc3-a0ee-5a765f1e3fe1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:24.923075 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8603ec9-0cbf-4dc3-a0ee-5a765f1e3fe1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:24.924616 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8603ec9-0cbf-4dc3-a0ee-5a765f1e3fe1 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:24.928426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:24.936712 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1311/2602] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:40:24 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:40:24.954693 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e989a-4a23-474b-a68a-7c50999b3416 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:25.024741 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e989a-4a23-474b-a68a-7c50999b3416 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:25.032771 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9aa813-8a05-4436-b5e5-77a595efa611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:25.033658 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9aa813-8a05-4436-b5e5-77a595efa611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:25.033946 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9aa813-8a05-4436-b5e5-77a595efa611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:25.034229 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9aa813-8a05-4436-b5e5-77a595efa611 admin admin] PATH_INFO: `/v3/users/9e6d3d00e2a24f84974d28da937779ff` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:25.035996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9aa813-8a05-4436-b5e5-77a595efa611 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e6d3d00e2a24f84974d28da937779ff)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:25.038083 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9aa813-8a05-4436-b5e5-77a595efa611 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:25.087483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-ff9aa813-8a05-4436-b5e5-77a595efa611 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e6d3d00e2a24f84974d28da937779ff'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:25.088199 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-ff9aa813-8a05-4436-b5e5-77a595efa611 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e6d3d00e2a24f84974d28da937779ff'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:25.095449 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-ff9aa813-8a05-4436-b5e5-77a595efa611 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e6d3d00e2a24f84974d28da937779ff'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:25.109858 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-ff9aa813-8a05-4436-b5e5-77a595efa611 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e6d3d00e2a24f84974d28da937779ff'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:25.112632 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:25.120835 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-ff9aa813-8a05-4436-b5e5-77a595efa611 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e6d3d00e2a24f84974d28da937779ff'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:25.129797 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1292/2603] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:40:24 2022] DELETE /identity/v3/users/9e6d3d00e2a24f84974d28da937779ff => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:25.135336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571382d0-af2f-458c-b478-3f2fff01ad97 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:25.135966 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571382d0-af2f-458c-b478-3f2fff01ad97 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:25.136646 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571382d0-af2f-458c-b478-3f2fff01ad97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:25.156765 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-571382d0-af2f-458c-b478-3f2fff01ad97 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:25.158020 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-571382d0-af2f-458c-b478-3f2fff01ad97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:25.247983 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1312/2604] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:25.260832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9aa813-8a05-4436-b5e5-77a595efa611 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:25.262439 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-ff9aa813-8a05-4436-b5e5-77a595efa611 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:25.326985 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9aa813-8a05-4436-b5e5-77a595efa611 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:25.331073 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb3b88-b8c1-42b4-9e31-3bfb88b50743 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:25.332647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb3b88-b8c1-42b4-9e31-3bfb88b50743 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:25.332934 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb3b88-b8c1-42b4-9e31-3bfb88b50743 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:25.333257 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb3b88-b8c1-42b4-9e31-3bfb88b50743 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:25.343863 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb3b88-b8c1-42b4-9e31-3bfb88b50743 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:25.345285 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb3b88-b8c1-42b4-9e31-3bfb88b50743 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:25.356733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1293/2605] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:40:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:40:25.725055 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571382d0-af2f-458c-b478-3f2fff01ad97 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:25.727067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-571382d0-af2f-458c-b478-3f2fff01ad97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:25.747613 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571382d0-af2f-458c-b478-3f2fff01ad97 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:25.756898 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9febff74-e286-413a-a3d8-a732977ecbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:25.758186 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febff74-e286-413a-a3d8-a732977ecbda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:25.758523 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febff74-e286-413a-a3d8-a732977ecbda admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:25.759011 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febff74-e286-413a-a3d8-a732977ecbda admin admin] PATH_INFO: `/v3/projects/b1c77009657c41fd9922e8828cfd7f38` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:25.760719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9febff74-e286-413a-a3d8-a732977ecbda admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1c77009657c41fd9922e8828cfd7f38)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:25.763530 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9febff74-e286-413a-a3d8-a732977ecbda admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:25.817464 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9febff74-e286-413a-a3d8-a732977ecbda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1c77009657c41fd9922e8828cfd7f38', 'request_id': 'req-9febff74-e286-413a-a3d8-a732977ecbda'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:25.823714 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9febff74-e286-413a-a3d8-a732977ecbda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1c77009657c41fd9922e8828cfd7f38', 'request_id': 'req-9febff74-e286-413a-a3d8-a732977ecbda'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:25.830720 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9febff74-e286-413a-a3d8-a732977ecbda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1c77009657c41fd9922e8828cfd7f38', 'request_id': 'req-9febff74-e286-413a-a3d8-a732977ecbda'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:25.845218 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9febff74-e286-413a-a3d8-a732977ecbda admin admin] The token cache is being invalidate because project b1c77009657c41fd9922e8828cfd7f38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:40:25.845513 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9febff74-e286-413a-a3d8-a732977ecbda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:25.846166 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-9febff74-e286-413a-a3d8-a732977ecbda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:25.848727 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1313/2606] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:40:25 2022] DELETE /identity/v3/projects/b1c77009657c41fd9922e8828cfd7f38 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:25.855043 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdaf41-b009-47ae-87b4-d99e6f79117e None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:25.855558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdaf41-b009-47ae-87b4-d99e6f79117e None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:25.856034 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdaf41-b009-47ae-87b4-d99e6f79117e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:25.867191 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9febff74-e286-413a-a3d8-a732977ecbda admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:25.869023 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-9febff74-e286-413a-a3d8-a732977ecbda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:25.877440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-65bdaf41-b009-47ae-87b4-d99e6f79117e None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:25.878691 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-65bdaf41-b009-47ae-87b4-d99e6f79117e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:25.953336 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1294/2607] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:25.954518 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9febff74-e286-413a-a3d8-a732977ecbda admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:25.961132 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461964c7-9fe5-4c95-8328-eb44cb2fc9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:25.962035 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461964c7-9fe5-4c95-8328-eb44cb2fc9ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:25.962384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461964c7-9fe5-4c95-8328-eb44cb2fc9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:25.962683 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461964c7-9fe5-4c95-8328-eb44cb2fc9ec admin admin] PATH_INFO: `/v3/users/79db7ba8e1fe42dbb4a98f6191d483f3` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:25.963012 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4f915-a1d6-415a-a9e2-cc9f021c8416 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:25.963309 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4f915-a1d6-415a-a9e2-cc9f021c8416 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:25.963596 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4f915-a1d6-415a-a9e2-cc9f021c8416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:25.964306 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461964c7-9fe5-4c95-8328-eb44cb2fc9ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=79db7ba8e1fe42dbb4a98f6191d483f3)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:25.966444 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461964c7-9fe5-4c95-8328-eb44cb2fc9ec admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:25.987337 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-adc4f915-a1d6-415a-a9e2-cc9f021c8416 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:25.988859 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-adc4f915-a1d6-415a-a9e2-cc9f021c8416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:25.996563 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:26.002372 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1295/2608] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:40:26.007703 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc4f915-a1d6-415a-a9e2-cc9f021c8416 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:26.027613 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc4f915-a1d6-415a-a9e2-cc9f021c8416 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:26.027926 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-461964c7-9fe5-4c95-8328-eb44cb2fc9ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79db7ba8e1fe42dbb4a98f6191d483f3'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:26.032491 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb7f08b-2bb0-4d85-93f2-42d769dbe1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:26.033171 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7f08b-2bb0-4d85-93f2-42d769dbe1c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:26.033653 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7f08b-2bb0-4d85-93f2-42d769dbe1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:26.033971 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7f08b-2bb0-4d85-93f2-42d769dbe1c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:26.035732 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb7f08b-2bb0-4d85-93f2-42d769dbe1c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:40:26.036045 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb7f08b-2bb0-4d85-93f2-42d769dbe1c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:26.037693 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb7f08b-2bb0-4d85-93f2-42d769dbe1c0 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:26.046296 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-461964c7-9fe5-4c95-8328-eb44cb2fc9ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79db7ba8e1fe42dbb4a98f6191d483f3'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:26.052118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1296/2609] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:40:26.056951 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb7f08b-2bb0-4d85-93f2-42d769dbe1c0 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:26.057215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-461964c7-9fe5-4c95-8328-eb44cb2fc9ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79db7ba8e1fe42dbb4a98f6191d483f3'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:26.058039 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-461964c7-9fe5-4c95-8328-eb44cb2fc9ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79db7ba8e1fe42dbb4a98f6191d483f3'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:26.058892 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-461964c7-9fe5-4c95-8328-eb44cb2fc9ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79db7ba8e1fe42dbb4a98f6191d483f3'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:26.067799 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb7f08b-2bb0-4d85-93f2-42d769dbe1c0 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:26.072583 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7a1f8-9640-465f-8609-20602d2a2b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:26.073242 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7a1f8-9640-465f-8609-20602d2a2b2e admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:26.073526 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7a1f8-9640-465f-8609-20602d2a2b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:26.073801 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7a1f8-9640-465f-8609-20602d2a2b2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:26.074504 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7a1f8-9640-465f-8609-20602d2a2b2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:26.076458 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7a1f8-9640-465f-8609-20602d2a2b2e admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:26.079152 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1314/2610] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:40:25 2022] DELETE /identity/v3/users/79db7ba8e1fe42dbb4a98f6191d483f3 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:26.087008 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37676984-5069-4822-9b43-ccf046bd88df None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:26.087461 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37676984-5069-4822-9b43-ccf046bd88df None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:26.087871 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37676984-5069-4822-9b43-ccf046bd88df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:26.111117 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1297/2611] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:40:26 2022] POST /identity/v3/projects => generated 391 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:40:26.115380 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-37676984-5069-4822-9b43-ccf046bd88df None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:26.115725 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7a1f8-9640-465f-8609-20602d2a2b2e admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:26.116853 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-37676984-5069-4822-9b43-ccf046bd88df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:26.117490 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-e0b7a1f8-9640-465f-8609-20602d2a2b2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:26.185749 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:26.188148 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1315/2612] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:40:26.194861 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7a1f8-9640-465f-8609-20602d2a2b2e admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:26.200399 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37676984-5069-4822-9b43-ccf046bd88df None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:26.201369 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc7302-1485-44bd-94ef-05b5225c188e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:26.202163 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc7302-1485-44bd-94ef-05b5225c188e admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:26.202341 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-37676984-5069-4822-9b43-ccf046bd88df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:26.202499 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc7302-1485-44bd-94ef-05b5225c188e admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:26.202725 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc7302-1485-44bd-94ef-05b5225c188e admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:26.203502 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc7302-1485-44bd-94ef-05b5225c188e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:26.205723 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc7302-1485-44bd-94ef-05b5225c188e admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:26.228772 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1298/2613] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:40:26 2022] POST /identity/v3/users => generated 296 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:40:26.232827 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc7302-1485-44bd-94ef-05b5225c188e admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:26.249813 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc7302-1485-44bd-94ef-05b5225c188e admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:26.257716 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19547665-f691-412f-aa51-070f8863b9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:26.258600 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19547665-f691-412f-aa51-070f8863b9c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:26.259028 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19547665-f691-412f-aa51-070f8863b9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:26.259365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19547665-f691-412f-aa51-070f8863b9c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:26.260362 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19547665-f691-412f-aa51-070f8863b9c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:26.262267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19547665-f691-412f-aa51-070f8863b9c0 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:26.275091 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1299/2614] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:40:26.279723 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19547665-f691-412f-aa51-070f8863b9c0 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:26.282491 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37676984-5069-4822-9b43-ccf046bd88df None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:26.288916 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2559794-a752-422c-a751-c74ccb43d90a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:26.290124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2559794-a752-422c-a751-c74ccb43d90a service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:26.290600 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2559794-a752-422c-a751-c74ccb43d90a service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:26.291052 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2559794-a752-422c-a751-c74ccb43d90a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:26.301515 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19547665-f691-412f-aa51-070f8863b9c0 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:26.307210 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2559794-a752-422c-a751-c74ccb43d90a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:26.307561 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e835d-e273-498c-a614-dba7a618efd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:26.308411 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e835d-e273-498c-a614-dba7a618efd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:26.308805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e835d-e273-498c-a614-dba7a618efd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:26.308984 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2559794-a752-422c-a751-c74ccb43d90a service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:26.309265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e835d-e273-498c-a614-dba7a618efd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:26.310105 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e835d-e273-498c-a614-dba7a618efd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:26.311892 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e835d-e273-498c-a614-dba7a618efd4 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:26.319878 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1316/2615] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:40:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:40:26.320841 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2559794-a752-422c-a751-c74ccb43d90a service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:26.322272 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d2559794-a752-422c-a751-c74ccb43d90a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:26.334237 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1300/2616] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:40:26.336045 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e835d-e273-498c-a614-dba7a618efd4 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:26.336509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2559794-a752-422c-a751-c74ccb43d90a service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:26.341604 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199af25-eee0-4d03-83ed-ce68ed49e167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:26.342327 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199af25-eee0-4d03-83ed-ce68ed49e167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:26.342617 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199af25-eee0-4d03-83ed-ce68ed49e167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:26.342899 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199af25-eee0-4d03-83ed-ce68ed49e167 admin admin] PATH_INFO: `/v3/users/bd8065d17fad415ab32d1fa2ce36180f` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:26.344161 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a199af25-eee0-4d03-83ed-ce68ed49e167 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd8065d17fad415ab32d1fa2ce36180f)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:26.346159 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a199af25-eee0-4d03-83ed-ce68ed49e167 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:26.354823 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e835d-e273-498c-a614-dba7a618efd4 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:26.361315 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ed4ed-1371-48d3-ae58-98f2559352e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:26.362494 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ed4ed-1371-48d3-ae58-98f2559352e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:26.364288 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ed4ed-1371-48d3-ae58-98f2559352e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:26.364288 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ed4ed-1371-48d3-ae58-98f2559352e3 admin admin] PATH_INFO: `/v3/projects/46d057e73848455d9a217e69fdb23e61/users/015f5f2d06694967b551ec12b3cfad74/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:26.387453 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ed4ed-1371-48d3-ae58-98f2559352e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46d057e73848455d9a217e69fdb23e61, user_id=015f5f2d06694967b551ec12b3cfad74, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:26.391824 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ed4ed-1371-48d3-ae58-98f2559352e3 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:26.413430 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1301/2617] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:40:26 2022] PUT /identity/v3/projects/46d057e73848455d9a217e69fdb23e61/users/015f5f2d06694967b551ec12b3cfad74/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:26.413740 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:26.427853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e50fe-4afe-4743-80c3-cca81bf03c8a None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:26.428373 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e50fe-4afe-4743-80c3-cca81bf03c8a None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:26.428871 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e50fe-4afe-4743-80c3-cca81bf03c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:26.430499 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a199af25-eee0-4d03-83ed-ce68ed49e167 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd8065d17fad415ab32d1fa2ce36180f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:26.454403 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a199af25-eee0-4d03-83ed-ce68ed49e167 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd8065d17fad415ab32d1fa2ce36180f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:26.466233 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-083e50fe-4afe-4743-80c3-cca81bf03c8a None None] MFA Rules not processed for user `015f5f2d06694967b551ec12b3cfad74`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:26.466635 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a199af25-eee0-4d03-83ed-ce68ed49e167 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd8065d17fad415ab32d1fa2ce36180f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:26.468199 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a199af25-eee0-4d03-83ed-ce68ed49e167 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd8065d17fad415ab32d1fa2ce36180f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:26.468416 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-083e50fe-4afe-4743-80c3-cca81bf03c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:26.468725 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a199af25-eee0-4d03-83ed-ce68ed49e167 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd8065d17fad415ab32d1fa2ce36180f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:26.494649 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1317/2618] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:40:26 2022] DELETE /identity/v3/users/bd8065d17fad415ab32d1fa2ce36180f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:26.499846 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed9736-6fde-43b7-914d-aea9d14381e5 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:26.500467 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed9736-6fde-43b7-914d-aea9d14381e5 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:26.500849 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed9736-6fde-43b7-914d-aea9d14381e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:26.562980 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-87ed9736-6fde-43b7-914d-aea9d14381e5 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:26.566521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-87ed9736-6fde-43b7-914d-aea9d14381e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:26.605367 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1302/2619] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:40:26.621579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e50fe-4afe-4743-80c3-cca81bf03c8a None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:26.623811 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-083e50fe-4afe-4743-80c3-cca81bf03c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:26.688442 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1318/2620] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:40:26.715987 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed9736-6fde-43b7-914d-aea9d14381e5 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:26.718204 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-87ed9736-6fde-43b7-914d-aea9d14381e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:26.810376 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e50fe-4afe-4743-80c3-cca81bf03c8a None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:26.830034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed9736-6fde-43b7-914d-aea9d14381e5 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:26.834286 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f28a651-c966-43dc-a157-da729f2893a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:26.834959 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28a651-c966-43dc-a157-da729f2893a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:26.835247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28a651-c966-43dc-a157-da729f2893a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:26.835530 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28a651-c966-43dc-a157-da729f2893a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:26.846608 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f28a651-c966-43dc-a157-da729f2893a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:26.849234 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f28a651-c966-43dc-a157-da729f2893a4 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:26.860972 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1319/2621] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:40:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:40:26.862153 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f28a651-c966-43dc-a157-da729f2893a4 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:26.869482 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-6f28a651-c966-43dc-a157-da729f2893a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:26.886714 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f28a651-c966-43dc-a157-da729f2893a4 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:26.892476 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f72ba5-a786-4d41-b459-47b2bd0d66ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:26.893507 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f72ba5-a786-4d41-b459-47b2bd0d66ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:26.894519 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f72ba5-a786-4d41-b459-47b2bd0d66ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:26.894805 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f72ba5-a786-4d41-b459-47b2bd0d66ad admin admin] PATH_INFO: `/v3/projects/8df9f2d2770741e8ba3497f769680b91` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:26.896050 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f72ba5-a786-4d41-b459-47b2bd0d66ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=8df9f2d2770741e8ba3497f769680b91)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:26.898586 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f72ba5-a786-4d41-b459-47b2bd0d66ad admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:26.905687 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:26.926122 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791f1b7-32a4-4b1b-8331-6831dc161153 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:26.928084 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791f1b7-32a4-4b1b-8331-6831dc161153 service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:26.928814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791f1b7-32a4-4b1b-8331-6831dc161153 service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:26.929391 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791f1b7-32a4-4b1b-8331-6831dc161153 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:26.962455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-78f72ba5-a786-4d41-b459-47b2bd0d66ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8df9f2d2770741e8ba3497f769680b91', 'request_id': 'req-78f72ba5-a786-4d41-b459-47b2bd0d66ad'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:26.974960 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:26.978157 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-78f72ba5-a786-4d41-b459-47b2bd0d66ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8df9f2d2770741e8ba3497f769680b91', 'request_id': 'req-78f72ba5-a786-4d41-b459-47b2bd0d66ad'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:26.996204 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-78f72ba5-a786-4d41-b459-47b2bd0d66ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8df9f2d2770741e8ba3497f769680b91', 'request_id': 'req-78f72ba5-a786-4d41-b459-47b2bd0d66ad'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:27.015360 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-78f72ba5-a786-4d41-b459-47b2bd0d66ad admin admin] The token cache is being invalidate because project 8df9f2d2770741e8ba3497f769680b91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:40:27.015769 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-78f72ba5-a786-4d41-b459-47b2bd0d66ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:27.016494 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-78f72ba5-a786-4d41-b459-47b2bd0d66ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:27.018314 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1320/2622] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:40:26 2022] DELETE /identity/v3/projects/8df9f2d2770741e8ba3497f769680b91 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:27.028464 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f72ba5-a786-4d41-b459-47b2bd0d66ad admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:27.029084 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3791f1b7-32a4-4b1b-8331-6831dc161153 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:27.030768 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3791f1b7-32a4-4b1b-8331-6831dc161153 service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:27.033233 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-78f72ba5-a786-4d41-b459-47b2bd0d66ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:27.042532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1303/2623] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:40:26 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:40:27.139276 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f72ba5-a786-4d41-b459-47b2bd0d66ad admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:27.146307 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f77d5-0be1-481a-a30d-c8236d7aba9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:27.147227 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f77d5-0be1-481a-a30d-c8236d7aba9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:27.147516 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f77d5-0be1-481a-a30d-c8236d7aba9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:27.147799 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f77d5-0be1-481a-a30d-c8236d7aba9c admin admin] PATH_INFO: `/v3/users/b264dcde78a34dcb827a9d4e35c15a41` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:27.149251 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f77d5-0be1-481a-a30d-c8236d7aba9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b264dcde78a34dcb827a9d4e35c15a41)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:27.151569 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f77d5-0be1-481a-a30d-c8236d7aba9c admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:27.215863 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-040f77d5-0be1-481a-a30d-c8236d7aba9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b264dcde78a34dcb827a9d4e35c15a41'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:27.233063 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-040f77d5-0be1-481a-a30d-c8236d7aba9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b264dcde78a34dcb827a9d4e35c15a41'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:27.245771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-040f77d5-0be1-481a-a30d-c8236d7aba9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b264dcde78a34dcb827a9d4e35c15a41'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:27.246683 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-040f77d5-0be1-481a-a30d-c8236d7aba9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b264dcde78a34dcb827a9d4e35c15a41'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:27.247497 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-040f77d5-0be1-481a-a30d-c8236d7aba9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b264dcde78a34dcb827a9d4e35c15a41'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:27.277184 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1321/2624] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:40:27 2022] DELETE /identity/v3/users/b264dcde78a34dcb827a9d4e35c15a41 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:27.325332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791f1b7-32a4-4b1b-8331-6831dc161153 service cinder] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:27.327619 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-3791f1b7-32a4-4b1b-8331-6831dc161153 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:27.343901 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791f1b7-32a4-4b1b-8331-6831dc161153 service cinder] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:27.461899 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027e4d2-e389-48dd-84fc-c1c9f8627c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:27.463483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027e4d2-e389-48dd-84fc-c1c9f8627c50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:27.463936 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027e4d2-e389-48dd-84fc-c1c9f8627c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:27.464443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027e4d2-e389-48dd-84fc-c1c9f8627c50 admin admin] PATH_INFO: `/v3/projects/1adfc0db703244038041f32fc483ad55` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:27.466486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a027e4d2-e389-48dd-84fc-c1c9f8627c50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1adfc0db703244038041f32fc483ad55)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:27.470198 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a027e4d2-e389-48dd-84fc-c1c9f8627c50 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:27.588759 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-a027e4d2-e389-48dd-84fc-c1c9f8627c50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1adfc0db703244038041f32fc483ad55', 'request_id': 'req-a027e4d2-e389-48dd-84fc-c1c9f8627c50'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:27.598460 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-a027e4d2-e389-48dd-84fc-c1c9f8627c50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1adfc0db703244038041f32fc483ad55', 'request_id': 'req-a027e4d2-e389-48dd-84fc-c1c9f8627c50'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:27.605962 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-a027e4d2-e389-48dd-84fc-c1c9f8627c50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1adfc0db703244038041f32fc483ad55', 'request_id': 'req-a027e4d2-e389-48dd-84fc-c1c9f8627c50'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:27.618193 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-a027e4d2-e389-48dd-84fc-c1c9f8627c50 admin admin] The token cache is being invalidate because project 1adfc0db703244038041f32fc483ad55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:40:27.618540 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-a027e4d2-e389-48dd-84fc-c1c9f8627c50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:27.619175 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-a027e4d2-e389-48dd-84fc-c1c9f8627c50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:27.625641 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1304/2625] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:40:27 2022] DELETE /identity/v3/projects/1adfc0db703244038041f32fc483ad55 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:27.627647 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ca27c-3a13-406b-aff6-46d83833d761 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:27.627945 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ca27c-3a13-406b-aff6-46d83833d761 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:27.628803 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ca27c-3a13-406b-aff6-46d83833d761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:27.653414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-730ca27c-3a13-406b-aff6-46d83833d761 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:27.654781 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-730ca27c-3a13-406b-aff6-46d83833d761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:27.658848 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027e4d2-e389-48dd-84fc-c1c9f8627c50 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:27.661349 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-a027e4d2-e389-48dd-84fc-c1c9f8627c50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:27.709342 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:27.747999 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1322/2626] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:27.749739 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027e4d2-e389-48dd-84fc-c1c9f8627c50 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:27.755541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85e0db-37e4-4df0-9f8f-a4c80c5fa125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:27.756458 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85e0db-37e4-4df0-9f8f-a4c80c5fa125 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:27.756830 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85e0db-37e4-4df0-9f8f-a4c80c5fa125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:27.757117 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85e0db-37e4-4df0-9f8f-a4c80c5fa125 admin admin] PATH_INFO: `/v3/projects/d5ae2b406d8b4a80a0de19c7d7f4c2df` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:27.758359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85e0db-37e4-4df0-9f8f-a4c80c5fa125 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5ae2b406d8b4a80a0de19c7d7f4c2df)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:27.759101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60453de-6113-42cc-a089-cff0f0ccbead None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:27.759794 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60453de-6113-42cc-a089-cff0f0ccbead None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:27.759794 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60453de-6113-42cc-a089-cff0f0ccbead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:27.760338 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85e0db-37e4-4df0-9f8f-a4c80c5fa125 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:27.768385 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:27.787583 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-e60453de-6113-42cc-a089-cff0f0ccbead None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:27.790064 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e60453de-6113-42cc-a089-cff0f0ccbead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:27.802854 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1323/2627] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:27.807635 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60453de-6113-42cc-a089-cff0f0ccbead None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:27.820595 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60453de-6113-42cc-a089-cff0f0ccbead None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:27.827927 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-5a85e0db-37e4-4df0-9f8f-a4c80c5fa125 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5ae2b406d8b4a80a0de19c7d7f4c2df', 'request_id': 'req-5a85e0db-37e4-4df0-9f8f-a4c80c5fa125'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:27.838291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-5a85e0db-37e4-4df0-9f8f-a4c80c5fa125 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5ae2b406d8b4a80a0de19c7d7f4c2df', 'request_id': 'req-5a85e0db-37e4-4df0-9f8f-a4c80c5fa125'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:27.845947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-5a85e0db-37e4-4df0-9f8f-a4c80c5fa125 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5ae2b406d8b4a80a0de19c7d7f4c2df', 'request_id': 'req-5a85e0db-37e4-4df0-9f8f-a4c80c5fa125'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:27.859863 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-5a85e0db-37e4-4df0-9f8f-a4c80c5fa125 admin admin] The token cache is being invalidate because project d5ae2b406d8b4a80a0de19c7d7f4c2df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:40:27.860163 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-5a85e0db-37e4-4df0-9f8f-a4c80c5fa125 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:27.860947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-5a85e0db-37e4-4df0-9f8f-a4c80c5fa125 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:27.862853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1305/2628] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:40:27 2022] DELETE /identity/v3/projects/d5ae2b406d8b4a80a0de19c7d7f4c2df => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:27.868849 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0df06d-95c2-43f4-91e8-fee0cac2c255 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:27.869153 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0df06d-95c2-43f4-91e8-fee0cac2c255 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:27.869914 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0df06d-95c2-43f4-91e8-fee0cac2c255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:27.889452 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-1e0df06d-95c2-43f4-91e8-fee0cac2c255 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:27.890845 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-1e0df06d-95c2-43f4-91e8-fee0cac2c255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:27.891531 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121516ca-988a-4ae1-8445-eb3abae4d5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:27.892375 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121516ca-988a-4ae1-8445-eb3abae4d5af admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:27.892728 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121516ca-988a-4ae1-8445-eb3abae4d5af admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:27.893149 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121516ca-988a-4ae1-8445-eb3abae4d5af admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:27.894767 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121516ca-988a-4ae1-8445-eb3abae4d5af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:40:27.895318 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121516ca-988a-4ae1-8445-eb3abae4d5af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:27.897963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121516ca-988a-4ae1-8445-eb3abae4d5af admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:27.902446 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1306/2629] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:40:27.910615 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6e772-856d-47e7-a624-0ab9bdd9a174 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:27.910922 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6e772-856d-47e7-a624-0ab9bdd9a174 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:27.911195 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6e772-856d-47e7-a624-0ab9bdd9a174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:27.915891 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1324/2630] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:40:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:40:27.917116 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121516ca-988a-4ae1-8445-eb3abae4d5af admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:27.919087 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-121516ca-988a-4ae1-8445-eb3abae4d5af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:27.931461 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-44d6e772-856d-47e7-a624-0ab9bdd9a174 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:27.932562 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121516ca-988a-4ae1-8445-eb3abae4d5af admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:27.932722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-44d6e772-856d-47e7-a624-0ab9bdd9a174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:27.937913 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e33ae8-5449-4695-81cf-e20652f014ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:27.938773 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e33ae8-5449-4695-81cf-e20652f014ac admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:27.940087 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e33ae8-5449-4695-81cf-e20652f014ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:27.941185 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e33ae8-5449-4695-81cf-e20652f014ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:27.942741 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e33ae8-5449-4695-81cf-e20652f014ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:27.943944 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1307/2631] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:40:27.946366 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e33ae8-5449-4695-81cf-e20652f014ac admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:27.949615 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6e772-856d-47e7-a624-0ab9bdd9a174 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:27.961614 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6e772-856d-47e7-a624-0ab9bdd9a174 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:27.967208 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e94007-f45d-466f-95e7-9c6c1c108dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:27.967955 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e94007-f45d-466f-95e7-9c6c1c108dcf admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:27.968298 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e94007-f45d-466f-95e7-9c6c1c108dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:27.968929 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e94007-f45d-466f-95e7-9c6c1c108dcf admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:27.969717 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e94007-f45d-466f-95e7-9c6c1c108dcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:40:27.969988 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e94007-f45d-466f-95e7-9c6c1c108dcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:27.971616 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e94007-f45d-466f-95e7-9c6c1c108dcf admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:27.985032 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1308/2632] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:40:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:40:27.989322 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1325/2633] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:40:27 2022] POST /identity/v3/projects => generated 399 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:40:27.990715 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e94007-f45d-466f-95e7-9c6c1c108dcf admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:27.993219 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e33ae8-5449-4695-81cf-e20652f014ac admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:28.002353 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e94007-f45d-466f-95e7-9c6c1c108dcf admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:28.083384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e33ae8-5449-4695-81cf-e20652f014ac admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:28.088338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10217cfd-fe3b-4ebe-baf7-ed703f0826c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:28.089023 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10217cfd-fe3b-4ebe-baf7-ed703f0826c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:28.089307 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10217cfd-fe3b-4ebe-baf7-ed703f0826c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:28.089585 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10217cfd-fe3b-4ebe-baf7-ed703f0826c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:28.090757 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10217cfd-fe3b-4ebe-baf7-ed703f0826c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:28.092661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10217cfd-fe3b-4ebe-baf7-ed703f0826c1 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:28.109288 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40516e19-8b95-4642-a5df-c045e24395ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:28.110087 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40516e19-8b95-4642-a5df-c045e24395ab admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:28.110425 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40516e19-8b95-4642-a5df-c045e24395ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:28.111058 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40516e19-8b95-4642-a5df-c045e24395ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:28.111957 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40516e19-8b95-4642-a5df-c045e24395ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:28.115144 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40516e19-8b95-4642-a5df-c045e24395ab admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:28.130123 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1326/2634] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:40:27 2022] POST /identity/v3/users => generated 300 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:40:28.131633 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10217cfd-fe3b-4ebe-baf7-ed703f0826c1 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:28.146408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10217cfd-fe3b-4ebe-baf7-ed703f0826c1 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:28.155113 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda29f2-b10e-4446-bf52-edd63b8ec365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:28.156402 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda29f2-b10e-4446-bf52-edd63b8ec365 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:28.156576 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1309/2635] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:40:27 2022] POST /identity/v3/projects => generated 371 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:40:28.156838 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda29f2-b10e-4446-bf52-edd63b8ec365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:28.157251 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda29f2-b10e-4446-bf52-edd63b8ec365 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:28.158150 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda29f2-b10e-4446-bf52-edd63b8ec365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:28.160446 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40516e19-8b95-4642-a5df-c045e24395ab admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:28.161291 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda29f2-b10e-4446-bf52-edd63b8ec365 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:28.178164 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1327/2636] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:40:28.182942 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda29f2-b10e-4446-bf52-edd63b8ec365 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:28.188968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:28.195092 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda29f2-b10e-4446-bf52-edd63b8ec365 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:28.261474 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:28.268415 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40516e19-8b95-4642-a5df-c045e24395ab admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:28.275808 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaccde14-d383-4f12-9d09-46d9e5c566af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:28.276730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccde14-d383-4f12-9d09-46d9e5c566af admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:28.277105 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccde14-d383-4f12-9d09-46d9e5c566af admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:28.277473 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccde14-d383-4f12-9d09-46d9e5c566af admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:28.278308 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaccde14-d383-4f12-9d09-46d9e5c566af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:28.278483 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a1e1c-e337-4919-b029-696bd6915f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:28.279047 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a1e1c-e337-4919-b029-696bd6915f43 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:28.279333 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a1e1c-e337-4919-b029-696bd6915f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:28.279614 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a1e1c-e337-4919-b029-696bd6915f43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:28.280379 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a1e1c-e337-4919-b029-696bd6915f43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:28.280864 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaccde14-d383-4f12-9d09-46d9e5c566af admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:28.281995 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a1e1c-e337-4919-b029-696bd6915f43 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:28.293836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1328/2637] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:40:28.298465 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a1e1c-e337-4919-b029-696bd6915f43 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:28.306277 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1310/2638] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:40:28 2022] POST /identity/v3/users => generated 286 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:40:28.311859 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaccde14-d383-4f12-9d09-46d9e5c566af admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:28.312016 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a1e1c-e337-4919-b029-696bd6915f43 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:28.317486 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398153d7-2f4d-4953-a0b3-252d3fda0eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:28.318536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398153d7-2f4d-4953-a0b3-252d3fda0eb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:28.318876 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398153d7-2f4d-4953-a0b3-252d3fda0eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:28.319255 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398153d7-2f4d-4953-a0b3-252d3fda0eb1 admin admin] PATH_INFO: `/v3/projects/b349553d3ea447fab3cd2da2124eebba/users/f7cbdb1551164b0caa54eb3fe76143f2/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:28.322926 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaccde14-d383-4f12-9d09-46d9e5c566af admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:28.328085 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6afbc-da36-4c56-bc87-3b027bddb008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:28.329036 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6afbc-da36-4c56-bc87-3b027bddb008 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:28.329307 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6afbc-da36-4c56-bc87-3b027bddb008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:28.329585 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6afbc-da36-4c56-bc87-3b027bddb008 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:28.330308 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a6afbc-da36-4c56-bc87-3b027bddb008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:28.333395 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a6afbc-da36-4c56-bc87-3b027bddb008 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:28.338168 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398153d7-2f4d-4953-a0b3-252d3fda0eb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b349553d3ea447fab3cd2da2124eebba, user_id=f7cbdb1551164b0caa54eb3fe76143f2, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:28.341059 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398153d7-2f4d-4953-a0b3-252d3fda0eb1 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:28.343538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1311/2639] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:40:28.352501 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6afbc-da36-4c56-bc87-3b027bddb008 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:28.364900 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6afbc-da36-4c56-bc87-3b027bddb008 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:28.366097 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1329/2640] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:40:28 2022] PUT /identity/v3/projects/b349553d3ea447fab3cd2da2124eebba/users/f7cbdb1551164b0caa54eb3fe76143f2/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:40:28.380934 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4589f-e461-4db1-82ec-a5b6beb5621a None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:28.381229 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4589f-e461-4db1-82ec-a5b6beb5621a None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:28.381514 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4589f-e461-4db1-82ec-a5b6beb5621a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:28.419886 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-e1b4589f-e461-4db1-82ec-a5b6beb5621a None None] MFA Rules not processed for user `f7cbdb1551164b0caa54eb3fe76143f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:28.421403 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e1b4589f-e461-4db1-82ec-a5b6beb5621a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:28.444035 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf79d67-da54-4ca6-ae66-93b511e680d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:28.444752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf79d67-da54-4ca6-ae66-93b511e680d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:28.445034 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf79d67-da54-4ca6-ae66-93b511e680d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:28.445310 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf79d67-da54-4ca6-ae66-93b511e680d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:28.446036 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf79d67-da54-4ca6-ae66-93b511e680d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:28.447631 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf79d67-da54-4ca6-ae66-93b511e680d3 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:28.459315 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1312/2641] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:40:28.463841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf79d67-da54-4ca6-ae66-93b511e680d3 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:28.474345 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf79d67-da54-4ca6-ae66-93b511e680d3 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:28.479895 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da66ce2a-02ba-4a7c-acc8-b5d5bc863d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:28.481065 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66ce2a-02ba-4a7c-acc8-b5d5bc863d88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:28.481491 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66ce2a-02ba-4a7c-acc8-b5d5bc863d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:28.481868 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66ce2a-02ba-4a7c-acc8-b5d5bc863d88 admin admin] PATH_INFO: `/v3/projects/c1f2700efc5d45119cc13a60bc0f25f0/users/be8b5c7b8f48419ebcae6a1913661afa/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:28.497191 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1330/2642] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:40:28.503943 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da66ce2a-02ba-4a7c-acc8-b5d5bc863d88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1f2700efc5d45119cc13a60bc0f25f0, user_id=be8b5c7b8f48419ebcae6a1913661afa, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:28.508483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da66ce2a-02ba-4a7c-acc8-b5d5bc863d88 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:28.518517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4589f-e461-4db1-82ec-a5b6beb5621a None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:28.519699 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e1b4589f-e461-4db1-82ec-a5b6beb5621a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:28.543515 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1313/2643] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:40:28 2022] PUT /identity/v3/projects/c1f2700efc5d45119cc13a60bc0f25f0/users/be8b5c7b8f48419ebcae6a1913661afa/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:28.551300 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cca020-e8f3-475e-ad86-1b62cd7468a1 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:28.551746 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cca020-e8f3-475e-ad86-1b62cd7468a1 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:28.552246 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cca020-e8f3-475e-ad86-1b62cd7468a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:28.582817 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-c9cca020-e8f3-475e-ad86-1b62cd7468a1 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:28.584910 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-c9cca020-e8f3-475e-ad86-1b62cd7468a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:28.677502 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4589f-e461-4db1-82ec-a5b6beb5621a None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:28.719994 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1314/2644] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:40:28.733680 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cca020-e8f3-475e-ad86-1b62cd7468a1 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:28.741845 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-c9cca020-e8f3-475e-ad86-1b62cd7468a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:28.765755 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:28.782993 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b23f3-61fa-4e20-a62a-33aacc8de320 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:28.784164 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b23f3-61fa-4e20-a62a-33aacc8de320 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:28.788440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b23f3-61fa-4e20-a62a-33aacc8de320 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:28.788440 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b23f3-61fa-4e20-a62a-33aacc8de320 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:28.827756 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:28.890573 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cca020-e8f3-475e-ad86-1b62cd7468a1 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:28.898787 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db5888-ae61-40b7-b534-f22ea18d42f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:28.900123 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db5888-ae61-40b7-b534-f22ea18d42f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:28.900123 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db5888-ae61-40b7-b534-f22ea18d42f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:28.900591 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db5888-ae61-40b7-b534-f22ea18d42f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:28.931202 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db5888-ae61-40b7-b534-f22ea18d42f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:28.933045 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db5888-ae61-40b7-b534-f22ea18d42f8 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:28.938935 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3b23f3-61fa-4e20-a62a-33aacc8de320 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:28.940321 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3b23f3-61fa-4e20-a62a-33aacc8de320 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:28.963286 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1315/2645] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:40:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:40:28.964800 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1331/2646] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:40:28 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:40:30.078790 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e179b-9c01-4b28-ac13-3b8112e1c745 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:30.079816 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e179b-9c01-4b28-ac13-3b8112e1c745 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:30.079816 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e179b-9c01-4b28-ac13-3b8112e1c745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:30.111751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-5c7e179b-9c01-4b28-ac13-3b8112e1c745 None None] MFA Rules not processed for user `be8b5c7b8f48419ebcae6a1913661afa`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:30.113669 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-5c7e179b-9c01-4b28-ac13-3b8112e1c745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:30.199518 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1316/2647] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:40:30.210648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b23f3-61fa-4e20-a62a-33aacc8de320 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:30.213073 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ae3b23f3-61fa-4e20-a62a-33aacc8de320 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:30.226804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b23f3-61fa-4e20-a62a-33aacc8de320 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:30.239966 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874674c6-e7d0-4532-b8d8-4d53d4cb9a17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:30.241064 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874674c6-e7d0-4532-b8d8-4d53d4cb9a17 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:30.241581 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874674c6-e7d0-4532-b8d8-4d53d4cb9a17 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:30.242068 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874674c6-e7d0-4532-b8d8-4d53d4cb9a17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:30.256588 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874674c6-e7d0-4532-b8d8-4d53d4cb9a17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:30.259041 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874674c6-e7d0-4532-b8d8-4d53d4cb9a17 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:30.272945 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1332/2648] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:40:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:40:33.756230 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f9322-65e6-4ee2-ac4a-c589046a4132 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:33.756817 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f9322-65e6-4ee2-ac4a-c589046a4132 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:33.756817 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f9322-65e6-4ee2-ac4a-c589046a4132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:33.774491 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-f02f9322-65e6-4ee2-ac4a-c589046a4132 None None] MFA Rules not processed for user `be8b5c7b8f48419ebcae6a1913661afa`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:33.775699 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-f02f9322-65e6-4ee2-ac4a-c589046a4132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:33.782629 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1317/2649] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:40:36.339391 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be0d00-26a7-4d2a-832d-86c21d9b521d None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:36.340224 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be0d00-26a7-4d2a-832d-86c21d9b521d None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:36.340224 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be0d00-26a7-4d2a-832d-86c21d9b521d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:36.359074 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-23be0d00-26a7-4d2a-832d-86c21d9b521d None None] MFA Rules not processed for user `733f8296ec3d4795a1140c1e1cb6eb95`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:36.360316 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-23be0d00-26a7-4d2a-832d-86c21d9b521d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:36.416832 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1333/2650] 10.222.0.16 () {58 vars in 967 bytes} [Mon May 23 12:40:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:36.429154 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f9322-65e6-4ee2-ac4a-c589046a4132 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:36.430538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-f02f9322-65e6-4ee2-ac4a-c589046a4132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:36.506077 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f9322-65e6-4ee2-ac4a-c589046a4132 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:36.511103 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa98b5a-af0e-444f-8f60-b71bfcfc7b10 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71fb9a79ac494635804b93d4a0048205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:36.511751 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa98b5a-af0e-444f-8f60-b71bfcfc7b10 service swift] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:36.512036 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa98b5a-af0e-444f-8f60-b71bfcfc7b10 service swift] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:36.512350 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa98b5a-af0e-444f-8f60-b71bfcfc7b10 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:36.521808 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa98b5a-af0e-444f-8f60-b71bfcfc7b10 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:36.522970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa98b5a-af0e-444f-8f60-b71bfcfc7b10 service swift] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:36.531258 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1318/2651] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 12:40:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 12:40:41.530604 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be0d00-26a7-4d2a-832d-86c21d9b521d None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:41.542713 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be0d00-26a7-4d2a-832d-86c21d9b521d None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:41.547419 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136cfbb-3e2a-4b37-8818-2794a5eaefc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:41.548345 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136cfbb-3e2a-4b37-8818-2794a5eaefc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:41.548630 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136cfbb-3e2a-4b37-8818-2794a5eaefc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:41.548911 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136cfbb-3e2a-4b37-8818-2794a5eaefc7 admin admin] PATH_INFO: `/v3/users/f7cbdb1551164b0caa54eb3fe76143f2` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:41.550739 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c136cfbb-3e2a-4b37-8818-2794a5eaefc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7cbdb1551164b0caa54eb3fe76143f2)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:41.552754 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c136cfbb-3e2a-4b37-8818-2794a5eaefc7 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:41.606959 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c136cfbb-3e2a-4b37-8818-2794a5eaefc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7cbdb1551164b0caa54eb3fe76143f2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:41.622325 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c136cfbb-3e2a-4b37-8818-2794a5eaefc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7cbdb1551164b0caa54eb3fe76143f2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:41.632553 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c136cfbb-3e2a-4b37-8818-2794a5eaefc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7cbdb1551164b0caa54eb3fe76143f2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:41.633209 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c136cfbb-3e2a-4b37-8818-2794a5eaefc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7cbdb1551164b0caa54eb3fe76143f2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:41.633823 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-c136cfbb-3e2a-4b37-8818-2794a5eaefc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7cbdb1551164b0caa54eb3fe76143f2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:41.648622 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1334/2652] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:40:41 2022] DELETE /identity/v3/users/f7cbdb1551164b0caa54eb3fe76143f2 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:41.653046 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc939aab-2251-46ee-9668-1e34184e0b6b None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:41.653609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc939aab-2251-46ee-9668-1e34184e0b6b None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:41.653609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc939aab-2251-46ee-9668-1e34184e0b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:41.672598 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-cc939aab-2251-46ee-9668-1e34184e0b6b None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:41.674104 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-cc939aab-2251-46ee-9668-1e34184e0b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:41.699224 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:41.752478 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1319/2653] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:41.763281 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136cfbb-3e2a-4b37-8818-2794a5eaefc7 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:41.764821 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c136cfbb-3e2a-4b37-8818-2794a5eaefc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:41.792971 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:41.829171 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136cfbb-3e2a-4b37-8818-2794a5eaefc7 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:41.833031 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e6f68c-58dc-465a-ab1d-1827e55b6636 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:41.833692 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e6f68c-58dc-465a-ab1d-1827e55b6636 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:41.833977 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e6f68c-58dc-465a-ab1d-1827e55b6636 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:41.834257 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e6f68c-58dc-465a-ab1d-1827e55b6636 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:41.844130 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e6f68c-58dc-465a-ab1d-1827e55b6636 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:41.845589 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e6f68c-58dc-465a-ab1d-1827e55b6636 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:41.856484 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1335/2654] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:40:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:40:42.150239 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc939aab-2251-46ee-9668-1e34184e0b6b None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:42.151695 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-cc939aab-2251-46ee-9668-1e34184e0b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:42.166845 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc939aab-2251-46ee-9668-1e34184e0b6b None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:42.171864 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff5688c-c72e-4772-af00-46a7668f94d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:42.172769 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5688c-c72e-4772-af00-46a7668f94d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:42.173057 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5688c-c72e-4772-af00-46a7668f94d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:42.173346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5688c-c72e-4772-af00-46a7668f94d3 admin admin] PATH_INFO: `/v3/projects/b349553d3ea447fab3cd2da2124eebba` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:42.174572 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff5688c-c72e-4772-af00-46a7668f94d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b349553d3ea447fab3cd2da2124eebba)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:42.176580 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff5688c-c72e-4772-af00-46a7668f94d3 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:42.226055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-fff5688c-c72e-4772-af00-46a7668f94d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b349553d3ea447fab3cd2da2124eebba', 'request_id': 'req-fff5688c-c72e-4772-af00-46a7668f94d3'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:42.233860 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-fff5688c-c72e-4772-af00-46a7668f94d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b349553d3ea447fab3cd2da2124eebba', 'request_id': 'req-fff5688c-c72e-4772-af00-46a7668f94d3'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:42.241017 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-fff5688c-c72e-4772-af00-46a7668f94d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b349553d3ea447fab3cd2da2124eebba', 'request_id': 'req-fff5688c-c72e-4772-af00-46a7668f94d3'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:42.252118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-fff5688c-c72e-4772-af00-46a7668f94d3 admin admin] The token cache is being invalidate because project b349553d3ea447fab3cd2da2124eebba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:40:42.252441 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-fff5688c-c72e-4772-af00-46a7668f94d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:42.254829 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-fff5688c-c72e-4772-af00-46a7668f94d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:42.256383 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1320/2655] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:40:42 2022] DELETE /identity/v3/projects/b349553d3ea447fab3cd2da2124eebba => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:40:50.285052 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e6f68c-58dc-465a-ab1d-1827e55b6636 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:50.286719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a0e6f68c-58dc-465a-ab1d-1827e55b6636 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:50.299325 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e6f68c-58dc-465a-ab1d-1827e55b6636 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:50.356498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fe5d5-47fe-4dca-b6f5-76eb8bbc18b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:50.357372 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fe5d5-47fe-4dca-b6f5-76eb8bbc18b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:50.357655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fe5d5-47fe-4dca-b6f5-76eb8bbc18b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:50.357932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fe5d5-47fe-4dca-b6f5-76eb8bbc18b4 admin admin] PATH_INFO: `/v3/users/015f5f2d06694967b551ec12b3cfad74` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:50.359099 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242fe5d5-47fe-4dca-b6f5-76eb8bbc18b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=015f5f2d06694967b551ec12b3cfad74)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:50.361054 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242fe5d5-47fe-4dca-b6f5-76eb8bbc18b4 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:50.407601 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-242fe5d5-47fe-4dca-b6f5-76eb8bbc18b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '015f5f2d06694967b551ec12b3cfad74'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:50.419240 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-242fe5d5-47fe-4dca-b6f5-76eb8bbc18b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '015f5f2d06694967b551ec12b3cfad74'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:50.427001 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-242fe5d5-47fe-4dca-b6f5-76eb8bbc18b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '015f5f2d06694967b551ec12b3cfad74'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:50.427826 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-242fe5d5-47fe-4dca-b6f5-76eb8bbc18b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '015f5f2d06694967b551ec12b3cfad74'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:50.428475 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-242fe5d5-47fe-4dca-b6f5-76eb8bbc18b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '015f5f2d06694967b551ec12b3cfad74'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:50.441104 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1336/2656] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:40:50 2022] DELETE /identity/v3/users/015f5f2d06694967b551ec12b3cfad74 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:50.446572 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b658d-6cc4-4aa1-bf6d-5c83933ebb54 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:50.447104 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b658d-6cc4-4aa1-bf6d-5c83933ebb54 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:50.447210 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b658d-6cc4-4aa1-bf6d-5c83933ebb54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:50.465339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-357b658d-6cc4-4aa1-bf6d-5c83933ebb54 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:50.466545 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-357b658d-6cc4-4aa1-bf6d-5c83933ebb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:50.524889 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1321/2657] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:40:50.534483 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fe5d5-47fe-4dca-b6f5-76eb8bbc18b4 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:50.536006 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-242fe5d5-47fe-4dca-b6f5-76eb8bbc18b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:50.585546 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:50.599585 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fe5d5-47fe-4dca-b6f5-76eb8bbc18b4 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:50.603237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ffce9d-473e-41ce-9dbf-81e8882e2b78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:50.603878 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ffce9d-473e-41ce-9dbf-81e8882e2b78 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:50.604160 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ffce9d-473e-41ce-9dbf-81e8882e2b78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:50.604464 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ffce9d-473e-41ce-9dbf-81e8882e2b78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:50.614261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ffce9d-473e-41ce-9dbf-81e8882e2b78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:50.615940 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ffce9d-473e-41ce-9dbf-81e8882e2b78 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:50.625966 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1337/2658] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:40:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:40:50.972049 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b658d-6cc4-4aa1-bf6d-5c83933ebb54 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:40:50.974116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-357b658d-6cc4-4aa1-bf6d-5c83933ebb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:50.993213 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b658d-6cc4-4aa1-bf6d-5c83933ebb54 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:40:50.999142 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93572e-d5cb-4c1c-8280-cf89a4162cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:40:51.000107 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93572e-d5cb-4c1c-8280-cf89a4162cea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:51.000420 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93572e-d5cb-4c1c-8280-cf89a4162cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:51.000700 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93572e-d5cb-4c1c-8280-cf89a4162cea admin admin] PATH_INFO: `/v3/projects/46d057e73848455d9a217e69fdb23e61` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:51.002128 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f93572e-d5cb-4c1c-8280-cf89a4162cea admin admin] RBAC: Authorizing `identity:delete_project(project_id=46d057e73848455d9a217e69fdb23e61)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:40:51.004375 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f93572e-d5cb-4c1c-8280-cf89a4162cea admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:40:51.050913 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:40:51.056324 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-3f93572e-d5cb-4c1c-8280-cf89a4162cea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46d057e73848455d9a217e69fdb23e61', 'request_id': 'req-3f93572e-d5cb-4c1c-8280-cf89a4162cea'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:51.063832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-3f93572e-d5cb-4c1c-8280-cf89a4162cea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46d057e73848455d9a217e69fdb23e61', 'request_id': 'req-3f93572e-d5cb-4c1c-8280-cf89a4162cea'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:51.071061 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-3f93572e-d5cb-4c1c-8280-cf89a4162cea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46d057e73848455d9a217e69fdb23e61', 'request_id': 'req-3f93572e-d5cb-4c1c-8280-cf89a4162cea'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:51.082022 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-3f93572e-d5cb-4c1c-8280-cf89a4162cea admin admin] The token cache is being invalidate because project 46d057e73848455d9a217e69fdb23e61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:40:51.082315 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-3f93572e-d5cb-4c1c-8280-cf89a4162cea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:51.083700 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-3f93572e-d5cb-4c1c-8280-cf89a4162cea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:40:51.085369 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1322/2659] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:40:50 2022] DELETE /identity/v3/projects/46d057e73848455d9a217e69fdb23e61 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:40:55.936780 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7282b6-7a63-4a02-a6ec-cb96e713e8da None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:40:55.937275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7282b6-7a63-4a02-a6ec-cb96e713e8da None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:40:55.937386 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7282b6-7a63-4a02-a6ec-cb96e713e8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:40:55.954718 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-5a7282b6-7a63-4a02-a6ec-cb96e713e8da None None] MFA Rules not processed for user `be8b5c7b8f48419ebcae6a1913661afa`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:40:55.955905 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5a7282b6-7a63-4a02-a6ec-cb96e713e8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:40:56.011487 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1338/2660] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:40:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:41:01.079954 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=8,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:06.003876 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=31 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:13.030285 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93572e-d5cb-4c1c-8280-cf89a4162cea admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:13.033597 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-3f93572e-d5cb-4c1c-8280-cf89a4162cea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:13.096665 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93572e-d5cb-4c1c-8280-cf89a4162cea admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:13.101103 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25800fbc-dde7-4323-aa7d-268c2a2c25c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:13.101845 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25800fbc-dde7-4323-aa7d-268c2a2c25c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:13.102705 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25800fbc-dde7-4323-aa7d-268c2a2c25c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:13.102992 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25800fbc-dde7-4323-aa7d-268c2a2c25c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:13.105228 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-25800fbc-dde7-4323-aa7d-268c2a2c25c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:13.169053 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25800fbc-dde7-4323-aa7d-268c2a2c25c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:13.170607 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25800fbc-dde7-4323-aa7d-268c2a2c25c4 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:13.181223 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1323/2661] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:41:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:41:13.554215 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98502f1-e419-4be0-b00a-fa1035447cbf None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:13.554749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98502f1-e419-4be0-b00a-fa1035447cbf None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:13.555953 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98502f1-e419-4be0-b00a-fa1035447cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:13.575405 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-e98502f1-e419-4be0-b00a-fa1035447cbf None None] MFA Rules not processed for user `733f8296ec3d4795a1140c1e1cb6eb95`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:13.576775 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e98502f1-e419-4be0-b00a-fa1035447cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:13.647261 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1339/2662] 10.222.0.16 () {58 vars in 1009 bytes} [Mon May 23 12:41:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:41:13.671029 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25800fbc-dde7-4323-aa7d-268c2a2c25c4 service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:13.672481 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-25800fbc-dde7-4323-aa7d-268c2a2c25c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:13.748243 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25800fbc-dde7-4323-aa7d-268c2a2c25c4 service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:13.753393 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358a846-ce0f-4672-8b21-f1d171b6b194 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71fb9a79ac494635804b93d4a0048205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:13.754095 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358a846-ce0f-4672-8b21-f1d171b6b194 service swift] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:13.754382 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358a846-ce0f-4672-8b21-f1d171b6b194 service swift] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:13.754752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358a846-ce0f-4672-8b21-f1d171b6b194 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:13.764829 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358a846-ce0f-4672-8b21-f1d171b6b194 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:13.766496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358a846-ce0f-4672-8b21-f1d171b6b194 service swift] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:13.776355 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1324/2663] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 12:41:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 12:41:17.343888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98502f1-e419-4be0-b00a-fa1035447cbf None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:17.345769 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e98502f1-e419-4be0-b00a-fa1035447cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:17.498955 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98502f1-e419-4be0-b00a-fa1035447cbf None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:17.505546 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3b9cc-37e5-4a25-a5a5-c5b93d4b5bc6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc0f137e0d24dd2a460689bc17a1b98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:17.506478 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3b9cc-37e5-4a25-a5a5-c5b93d4b5bc6 service placement] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:17.506954 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3b9cc-37e5-4a25-a5a5-c5b93d4b5bc6 service placement] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:17.508874 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3b9cc-37e5-4a25-a5a5-c5b93d4b5bc6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:17.511522 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-15d3b9cc-37e5-4a25-a5a5-c5b93d4b5bc6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:17.532803 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3b9cc-37e5-4a25-a5a5-c5b93d4b5bc6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:17.534976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3b9cc-37e5-4a25-a5a5-c5b93d4b5bc6 service placement] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:17.558086 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1340/2664] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:41:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:41:18.854056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358a846-ce0f-4672-8b21-f1d171b6b194 service swift] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:18.865636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358a846-ce0f-4672-8b21-f1d171b6b194 service swift] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:18.871527 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e96e0-3611-44d1-8606-497928b6da07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:18.872692 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e96e0-3611-44d1-8606-497928b6da07 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:18.873786 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e96e0-3611-44d1-8606-497928b6da07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:18.879615 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e96e0-3611-44d1-8606-497928b6da07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:18.883670 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-243e96e0-3611-44d1-8606-497928b6da07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:18.985322 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:19.009782 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243e96e0-3611-44d1-8606-497928b6da07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:19.011543 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243e96e0-3611-44d1-8606-497928b6da07 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:19.034551 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1325/2665] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:41:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:41:23.420147 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3b9cc-37e5-4a25-a5a5-c5b93d4b5bc6 service placement] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:23.431291 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3b9cc-37e5-4a25-a5a5-c5b93d4b5bc6 service placement] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:23.438739 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d28ac-30b7-49b6-a3d7-d72fd478d44d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:23.439683 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d28ac-30b7-49b6-a3d7-d72fd478d44d service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:23.440147 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d28ac-30b7-49b6-a3d7-d72fd478d44d service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:23.440624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d28ac-30b7-49b6-a3d7-d72fd478d44d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:23.442795 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5d5d28ac-30b7-49b6-a3d7-d72fd478d44d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:23.459529 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5d28ac-30b7-49b6-a3d7-d72fd478d44d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:23.461302 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5d28ac-30b7-49b6-a3d7-d72fd478d44d service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:23.489600 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1341/2666] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:41:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:41:28.561719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e96e0-3611-44d1-8606-497928b6da07 service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:28.567159 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-243e96e0-3611-44d1-8606-497928b6da07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:28.605349 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e96e0-3611-44d1-8606-497928b6da07 service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:28.622483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac3dbd-1874-4bd8-98f3-7d2b244846ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:28.624775 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac3dbd-1874-4bd8-98f3-7d2b244846ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:28.625328 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac3dbd-1874-4bd8-98f3-7d2b244846ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:28.625930 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac3dbd-1874-4bd8-98f3-7d2b244846ed admin admin] PATH_INFO: `/v3/users/851454212d14438d80bf8e193fea268e` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:28.628064 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac3dbd-1874-4bd8-98f3-7d2b244846ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=851454212d14438d80bf8e193fea268e)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:28.631392 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ac3dbd-1874-4bd8-98f3-7d2b244846ed admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:28.661658 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d28ac-30b7-49b6-a3d7-d72fd478d44d service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:28.663197 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5d5d28ac-30b7-49b6-a3d7-d72fd478d44d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:28.678244 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d28ac-30b7-49b6-a3d7-d72fd478d44d service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:28.692425 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f9c7e-3dc5-441b-9550-4adfeaf57fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:28.693901 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f9c7e-3dc5-441b-9550-4adfeaf57fbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:28.694407 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f9c7e-3dc5-441b-9550-4adfeaf57fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:28.694908 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f9c7e-3dc5-441b-9550-4adfeaf57fbd admin admin] PATH_INFO: `/v3/users/921861792ac246aa8772a636d45489c8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:28.699521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f9c7e-3dc5-441b-9550-4adfeaf57fbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=921861792ac246aa8772a636d45489c8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:28.704948 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8f9c7e-3dc5-441b-9550-4adfeaf57fbd admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:28.752754 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-97ac3dbd-1874-4bd8-98f3-7d2b244846ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '851454212d14438d80bf8e193fea268e'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:28.754429 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-97ac3dbd-1874-4bd8-98f3-7d2b244846ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '851454212d14438d80bf8e193fea268e'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:28.771802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-97ac3dbd-1874-4bd8-98f3-7d2b244846ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '851454212d14438d80bf8e193fea268e'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:28.794021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-97ac3dbd-1874-4bd8-98f3-7d2b244846ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '851454212d14438d80bf8e193fea268e'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:28.807938 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1d8f9c7e-3dc5-441b-9550-4adfeaf57fbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '921861792ac246aa8772a636d45489c8'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:28.808162 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-97ac3dbd-1874-4bd8-98f3-7d2b244846ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '851454212d14438d80bf8e193fea268e'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:28.825749 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1326/2667] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:41:28 2022] DELETE /identity/v3/users/851454212d14438d80bf8e193fea268e => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:28.844799 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1d8f9c7e-3dc5-441b-9550-4adfeaf57fbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '921861792ac246aa8772a636d45489c8'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:28.856206 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1d8f9c7e-3dc5-441b-9550-4adfeaf57fbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '921861792ac246aa8772a636d45489c8'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:28.857270 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1d8f9c7e-3dc5-441b-9550-4adfeaf57fbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '921861792ac246aa8772a636d45489c8'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:28.858159 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1d8f9c7e-3dc5-441b-9550-4adfeaf57fbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '921861792ac246aa8772a636d45489c8'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:28.865172 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:28.915831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1342/2668] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:41:28 2022] DELETE /identity/v3/users/921861792ac246aa8772a636d45489c8 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:29.166663 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac3dbd-1874-4bd8-98f3-7d2b244846ed admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:29.169621 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-97ac3dbd-1874-4bd8-98f3-7d2b244846ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:29.212270 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f9c7e-3dc5-441b-9550-4adfeaf57fbd admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:29.214051 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1d8f9c7e-3dc5-441b-9550-4adfeaf57fbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:29.303606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac3dbd-1874-4bd8-98f3-7d2b244846ed admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:29.331528 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8f9c7e-3dc5-441b-9550-4adfeaf57fbd admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:29.350117 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a501c56-eb34-46fe-9eac-d2b8d4dd7bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:29.351132 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a501c56-eb34-46fe-9eac-d2b8d4dd7bc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:29.351628 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a501c56-eb34-46fe-9eac-d2b8d4dd7bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:29.352587 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a501c56-eb34-46fe-9eac-d2b8d4dd7bc1 admin admin] PATH_INFO: `/v3/projects/835f4c15c26e47b7ba55aca85bfb66a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:29.353909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a501c56-eb34-46fe-9eac-d2b8d4dd7bc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=835f4c15c26e47b7ba55aca85bfb66a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:29.357010 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a501c56-eb34-46fe-9eac-d2b8d4dd7bc1 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:29.380538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:29.390141 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4528eb9b-babc-4f8b-b6ca-6366f3a971a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:29.391725 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528eb9b-babc-4f8b-b6ca-6366f3a971a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:29.392427 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528eb9b-babc-4f8b-b6ca-6366f3a971a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:29.393229 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4528eb9b-babc-4f8b-b6ca-6366f3a971a7 admin admin] PATH_INFO: `/v3/projects/0502b9ef01e2460893cfc855d44db0c7` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:29.398646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4528eb9b-babc-4f8b-b6ca-6366f3a971a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0502b9ef01e2460893cfc855d44db0c7)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:29.401889 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4528eb9b-babc-4f8b-b6ca-6366f3a971a7 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:29.440770 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1a501c56-eb34-46fe-9eac-d2b8d4dd7bc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '835f4c15c26e47b7ba55aca85bfb66a8', 'request_id': 'req-1a501c56-eb34-46fe-9eac-d2b8d4dd7bc1'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:29.449590 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1a501c56-eb34-46fe-9eac-d2b8d4dd7bc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '835f4c15c26e47b7ba55aca85bfb66a8', 'request_id': 'req-1a501c56-eb34-46fe-9eac-d2b8d4dd7bc1'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:29.458092 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1a501c56-eb34-46fe-9eac-d2b8d4dd7bc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '835f4c15c26e47b7ba55aca85bfb66a8', 'request_id': 'req-1a501c56-eb34-46fe-9eac-d2b8d4dd7bc1'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:29.478979 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1a501c56-eb34-46fe-9eac-d2b8d4dd7bc1 admin admin] The token cache is being invalidate because project 835f4c15c26e47b7ba55aca85bfb66a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:41:29.479305 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1a501c56-eb34-46fe-9eac-d2b8d4dd7bc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:29.480017 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-4528eb9b-babc-4f8b-b6ca-6366f3a971a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0502b9ef01e2460893cfc855d44db0c7', 'request_id': 'req-4528eb9b-babc-4f8b-b6ca-6366f3a971a7'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:29.480146 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-1a501c56-eb34-46fe-9eac-d2b8d4dd7bc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:29.485005 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1343/2669] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:41:29 2022] DELETE /identity/v3/projects/835f4c15c26e47b7ba55aca85bfb66a8 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:29.485985 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a501c56-eb34-46fe-9eac-d2b8d4dd7bc1 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:29.488569 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1a501c56-eb34-46fe-9eac-d2b8d4dd7bc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:29.500600 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-4528eb9b-babc-4f8b-b6ca-6366f3a971a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0502b9ef01e2460893cfc855d44db0c7', 'request_id': 'req-4528eb9b-babc-4f8b-b6ca-6366f3a971a7'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:29.514222 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-4528eb9b-babc-4f8b-b6ca-6366f3a971a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0502b9ef01e2460893cfc855d44db0c7', 'request_id': 'req-4528eb9b-babc-4f8b-b6ca-6366f3a971a7'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:29.535960 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-4528eb9b-babc-4f8b-b6ca-6366f3a971a7 admin admin] The token cache is being invalidate because project 0502b9ef01e2460893cfc855d44db0c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:41:29.536374 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-4528eb9b-babc-4f8b-b6ca-6366f3a971a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:29.537300 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-4528eb9b-babc-4f8b-b6ca-6366f3a971a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:29.544229 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1327/2670] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:41:29 2022] DELETE /identity/v3/projects/0502b9ef01e2460893cfc855d44db0c7 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:29.547005 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f26b5-6bbe-4573-8377-4b09e558cdaf None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:29.547590 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f26b5-6bbe-4573-8377-4b09e558cdaf None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:29.547977 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f26b5-6bbe-4573-8377-4b09e558cdaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:29.574844 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-069f26b5-6bbe-4573-8377-4b09e558cdaf None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:29.576541 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-069f26b5-6bbe-4573-8377-4b09e558cdaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:29.588938 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a501c56-eb34-46fe-9eac-d2b8d4dd7bc1 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:29.631595 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:29.654505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1328/2671] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:41:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:41:29.663538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a9470-c03b-4fb5-b422-fd13a9eded10 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:29.665249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a9470-c03b-4fb5-b422-fd13a9eded10 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:29.665249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a9470-c03b-4fb5-b422-fd13a9eded10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:29.685592 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932c14a-600a-4513-aef8-ad5784ade91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:29.687122 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932c14a-600a-4513-aef8-ad5784ade91d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:29.687565 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932c14a-600a-4513-aef8-ad5784ade91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:29.687998 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932c14a-600a-4513-aef8-ad5784ade91d admin admin] PATH_INFO: `/v3/users/d787695b76ed4b09b744b3ac69fcbe4d` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:29.690010 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932c14a-600a-4513-aef8-ad5784ade91d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d787695b76ed4b09b744b3ac69fcbe4d)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:29.692237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932c14a-600a-4513-aef8-ad5784ade91d admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:29.692981 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-779a9470-c03b-4fb5-b422-fd13a9eded10 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:29.694848 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-779a9470-c03b-4fb5-b422-fd13a9eded10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:29.716322 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1329/2672] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:41:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:41:29.720152 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a9470-c03b-4fb5-b422-fd13a9eded10 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:29.732398 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a9470-c03b-4fb5-b422-fd13a9eded10 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:29.738616 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5fb223-ebeb-48f8-92ca-e16d8beb4c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:29.739292 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5fb223-ebeb-48f8-92ca-e16d8beb4c3d admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:29.739580 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5fb223-ebeb-48f8-92ca-e16d8beb4c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:29.739858 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5fb223-ebeb-48f8-92ca-e16d8beb4c3d admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:29.740686 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5fb223-ebeb-48f8-92ca-e16d8beb4c3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:41:29.741002 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5fb223-ebeb-48f8-92ca-e16d8beb4c3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:29.743221 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5fb223-ebeb-48f8-92ca-e16d8beb4c3d admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:29.764565 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1330/2673] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:41:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:41:29.766151 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5fb223-ebeb-48f8-92ca-e16d8beb4c3d admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:29.781776 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5fb223-ebeb-48f8-92ca-e16d8beb4c3d admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:29.797067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-7932c14a-600a-4513-aef8-ad5784ade91d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd787695b76ed4b09b744b3ac69fcbe4d'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:29.797624 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efae607-5eb7-4e70-9fba-ecdfa0cf357c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:29.799707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efae607-5eb7-4e70-9fba-ecdfa0cf357c admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:29.799707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efae607-5eb7-4e70-9fba-ecdfa0cf357c admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:29.799707 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efae607-5eb7-4e70-9fba-ecdfa0cf357c admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:29.805161 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efae607-5eb7-4e70-9fba-ecdfa0cf357c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:29.805161 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efae607-5eb7-4e70-9fba-ecdfa0cf357c admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:29.821854 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-7932c14a-600a-4513-aef8-ad5784ade91d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd787695b76ed4b09b744b3ac69fcbe4d'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:29.834828 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-7932c14a-600a-4513-aef8-ad5784ade91d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd787695b76ed4b09b744b3ac69fcbe4d'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:29.837918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-7932c14a-600a-4513-aef8-ad5784ade91d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd787695b76ed4b09b744b3ac69fcbe4d'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:29.838836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-7932c14a-600a-4513-aef8-ad5784ade91d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd787695b76ed4b09b744b3ac69fcbe4d'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:29.845650 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1331/2674] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:41:29 2022] POST /identity/v3/projects => generated 373 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:41:29.849618 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efae607-5eb7-4e70-9fba-ecdfa0cf357c admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:29.854484 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-7efae607-5eb7-4e70-9fba-ecdfa0cf357c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:29.872083 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1344/2675] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:41:29 2022] DELETE /identity/v3/users/d787695b76ed4b09b744b3ac69fcbe4d => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:41:29.938726 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:29.977425 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efae607-5eb7-4e70-9fba-ecdfa0cf357c admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:30.068871 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08965208-b83a-4b34-8434-532e4f7f4f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:30.069872 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08965208-b83a-4b34-8434-532e4f7f4f63 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:30.071415 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08965208-b83a-4b34-8434-532e4f7f4f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:30.071548 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08965208-b83a-4b34-8434-532e4f7f4f63 admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:30.072835 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08965208-b83a-4b34-8434-532e4f7f4f63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:30.076630 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08965208-b83a-4b34-8434-532e4f7f4f63 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:30.125491 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1332/2676] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:41:29 2022] POST /identity/v3/users => generated 287 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:41:30.131395 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932c14a-600a-4513-aef8-ad5784ade91d admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:30.148880 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08965208-b83a-4b34-8434-532e4f7f4f63 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:30.151492 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-08965208-b83a-4b34-8434-532e4f7f4f63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:30.156358 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932c14a-600a-4513-aef8-ad5784ade91d admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:30.166985 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08965208-b83a-4b34-8434-532e4f7f4f63 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:30.168264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443b956-05ba-4cc4-83e9-0b6492113148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:30.169428 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443b956-05ba-4cc4-83e9-0b6492113148 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:30.170015 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443b956-05ba-4cc4-83e9-0b6492113148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:30.170581 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1443b956-05ba-4cc4-83e9-0b6492113148 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:30.172116 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1443b956-05ba-4cc4-83e9-0b6492113148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:30.174117 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1443b956-05ba-4cc4-83e9-0b6492113148 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:30.187161 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a30bc2-2883-4f15-9a35-49bb8c05cb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:30.188193 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a30bc2-2883-4f15-9a35-49bb8c05cb9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:30.188650 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a30bc2-2883-4f15-9a35-49bb8c05cb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:30.189270 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a30bc2-2883-4f15-9a35-49bb8c05cb9c admin admin] PATH_INFO: `/v3/projects/48239413b5fe44c3905c37d9a647a41b` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:30.191025 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1345/2677] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:41:30 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:41:30.193365 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a30bc2-2883-4f15-9a35-49bb8c05cb9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=48239413b5fe44c3905c37d9a647a41b)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:30.195105 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443b956-05ba-4cc4-83e9-0b6492113148 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:30.197765 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a30bc2-2883-4f15-9a35-49bb8c05cb9c admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:30.217109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1443b956-05ba-4cc4-83e9-0b6492113148 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:30.223179 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c52b80-fc34-49cb-b91b-14ac860dc5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:30.224227 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c52b80-fc34-49cb-b91b-14ac860dc5cb admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:30.224818 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c52b80-fc34-49cb-b91b-14ac860dc5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:30.225488 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c52b80-fc34-49cb-b91b-14ac860dc5cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:30.226510 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c52b80-fc34-49cb-b91b-14ac860dc5cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:30.228385 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c52b80-fc34-49cb-b91b-14ac860dc5cb admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:30.245096 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1346/2678] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:41:30 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:41:30.249351 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c52b80-fc34-49cb-b91b-14ac860dc5cb admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:30.265215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-a6a30bc2-2883-4f15-9a35-49bb8c05cb9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48239413b5fe44c3905c37d9a647a41b', 'request_id': 'req-a6a30bc2-2883-4f15-9a35-49bb8c05cb9c'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:30.278097 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-a6a30bc2-2883-4f15-9a35-49bb8c05cb9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48239413b5fe44c3905c37d9a647a41b', 'request_id': 'req-a6a30bc2-2883-4f15-9a35-49bb8c05cb9c'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:30.283436 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c52b80-fc34-49cb-b91b-14ac860dc5cb admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:30.295234 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-a6a30bc2-2883-4f15-9a35-49bb8c05cb9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48239413b5fe44c3905c37d9a647a41b', 'request_id': 'req-a6a30bc2-2883-4f15-9a35-49bb8c05cb9c'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:30.312649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-a6a30bc2-2883-4f15-9a35-49bb8c05cb9c admin admin] The token cache is being invalidate because project 48239413b5fe44c3905c37d9a647a41b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:41:30.313594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-a6a30bc2-2883-4f15-9a35-49bb8c05cb9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:30.314410 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-a6a30bc2-2883-4f15-9a35-49bb8c05cb9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:30.316682 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1333/2679] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:41:30 2022] DELETE /identity/v3/projects/48239413b5fe44c3905c37d9a647a41b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:30.415727 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ccf93-9cef-44d5-828f-0a00b1889698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:30.417930 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ccf93-9cef-44d5-828f-0a00b1889698 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:30.421504 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ccf93-9cef-44d5-828f-0a00b1889698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:30.422322 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ccf93-9cef-44d5-828f-0a00b1889698 admin admin] PATH_INFO: `/v3/projects/d7480570ee8b45ac809e2a4aadf2b779/users/ce434a6f39ed4d65bd026f4f6323f996/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:30.459969 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ccf93-9cef-44d5-828f-0a00b1889698 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7480570ee8b45ac809e2a4aadf2b779, user_id=ce434a6f39ed4d65bd026f4f6323f996, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:30.469471 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ccf93-9cef-44d5-828f-0a00b1889698 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:30.512385 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1347/2680] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:41:30 2022] PUT /identity/v3/projects/d7480570ee8b45ac809e2a4aadf2b779/users/ce434a6f39ed4d65bd026f4f6323f996/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:30.523205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50abaf4d-2e14-466d-8c98-6c9496d4759b None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:30.524192 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50abaf4d-2e14-466d-8c98-6c9496d4759b None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:30.524563 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50abaf4d-2e14-466d-8c98-6c9496d4759b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:30.572079 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-50abaf4d-2e14-466d-8c98-6c9496d4759b None None] MFA Rules not processed for user `ce434a6f39ed4d65bd026f4f6323f996`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:30.573698 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-50abaf4d-2e14-466d-8c98-6c9496d4759b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:30.661960 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:30.696952 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1334/2681] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:41:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:41:30.698087 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ccf93-9cef-44d5-828f-0a00b1889698 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:30.699955 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-237ccf93-9cef-44d5-828f-0a00b1889698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:30.718969 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:30.817095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ccf93-9cef-44d5-828f-0a00b1889698 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:30.825597 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd694a74-8d24-442b-a41f-54515c40879d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:30.826666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd694a74-8d24-442b-a41f-54515c40879d service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:30.827150 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd694a74-8d24-442b-a41f-54515c40879d service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:30.827626 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd694a74-8d24-442b-a41f-54515c40879d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:30.848004 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd694a74-8d24-442b-a41f-54515c40879d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:30.850515 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd694a74-8d24-442b-a41f-54515c40879d service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:30.864975 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1348/2682] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:41:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:41:31.148031 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50abaf4d-2e14-466d-8c98-6c9496d4759b None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:31.150556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-50abaf4d-2e14-466d-8c98-6c9496d4759b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:31.264037 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50abaf4d-2e14-466d-8c98-6c9496d4759b None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:31.269895 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d7868-15fe-40e8-abce-04c08351c06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:31.270824 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d7868-15fe-40e8-abce-04c08351c06c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:31.271115 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d7868-15fe-40e8-abce-04c08351c06c admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:31.271399 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d7868-15fe-40e8-abce-04c08351c06c admin admin] PATH_INFO: `/v3/users/ce434a6f39ed4d65bd026f4f6323f996` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:31.272699 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d7868-15fe-40e8-abce-04c08351c06c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce434a6f39ed4d65bd026f4f6323f996)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:31.274833 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d7868-15fe-40e8-abce-04c08351c06c admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:31.372843 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-a15d7868-15fe-40e8-abce-04c08351c06c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce434a6f39ed4d65bd026f4f6323f996'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:31.373561 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-a15d7868-15fe-40e8-abce-04c08351c06c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce434a6f39ed4d65bd026f4f6323f996'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:31.383467 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-a15d7868-15fe-40e8-abce-04c08351c06c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce434a6f39ed4d65bd026f4f6323f996'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:31.400748 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-a15d7868-15fe-40e8-abce-04c08351c06c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce434a6f39ed4d65bd026f4f6323f996'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:31.412235 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-a15d7868-15fe-40e8-abce-04c08351c06c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce434a6f39ed4d65bd026f4f6323f996'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:31.424501 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1335/2683] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:41:31 2022] DELETE /identity/v3/users/ce434a6f39ed4d65bd026f4f6323f996 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:31.431333 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c31b33-0381-483e-9f44-cbb83395decc None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:31.431851 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c31b33-0381-483e-9f44-cbb83395decc None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:31.432350 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c31b33-0381-483e-9f44-cbb83395decc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:31.452824 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-f8c31b33-0381-483e-9f44-cbb83395decc None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:31.453711 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-f8c31b33-0381-483e-9f44-cbb83395decc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:31.521944 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1349/2684] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:41:31.533010 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d7868-15fe-40e8-abce-04c08351c06c admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:31.534467 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-a15d7868-15fe-40e8-abce-04c08351c06c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:31.614559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d7868-15fe-40e8-abce-04c08351c06c admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:31.614559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f48917-7dbc-4112-a6ae-737af58bcb58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:31.614559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f48917-7dbc-4112-a6ae-737af58bcb58 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:31.614559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f48917-7dbc-4112-a6ae-737af58bcb58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:31.615146 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f48917-7dbc-4112-a6ae-737af58bcb58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:31.642141 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f48917-7dbc-4112-a6ae-737af58bcb58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:31.643628 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f48917-7dbc-4112-a6ae-737af58bcb58 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:31.661615 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1336/2685] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:41:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:41:32.037899 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c31b33-0381-483e-9f44-cbb83395decc None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:32.039677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-f8c31b33-0381-483e-9f44-cbb83395decc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:32.070737 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c31b33-0381-483e-9f44-cbb83395decc None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:32.078153 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a608c2-2d45-469e-a76a-4930b73718fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:32.079012 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a608c2-2d45-469e-a76a-4930b73718fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:32.079277 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a608c2-2d45-469e-a76a-4930b73718fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:32.079812 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a608c2-2d45-469e-a76a-4930b73718fa admin admin] PATH_INFO: `/v3/projects/d7480570ee8b45ac809e2a4aadf2b779` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:32.081102 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a608c2-2d45-469e-a76a-4930b73718fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7480570ee8b45ac809e2a4aadf2b779)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:32.083019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a608c2-2d45-469e-a76a-4930b73718fa admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:32.155981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-65a608c2-2d45-469e-a76a-4930b73718fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7480570ee8b45ac809e2a4aadf2b779', 'request_id': 'req-65a608c2-2d45-469e-a76a-4930b73718fa'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:32.162692 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-65a608c2-2d45-469e-a76a-4930b73718fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7480570ee8b45ac809e2a4aadf2b779', 'request_id': 'req-65a608c2-2d45-469e-a76a-4930b73718fa'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:32.170503 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-65a608c2-2d45-469e-a76a-4930b73718fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7480570ee8b45ac809e2a4aadf2b779', 'request_id': 'req-65a608c2-2d45-469e-a76a-4930b73718fa'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:32.187917 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-65a608c2-2d45-469e-a76a-4930b73718fa admin admin] The token cache is being invalidate because project d7480570ee8b45ac809e2a4aadf2b779 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:41:32.188249 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-65a608c2-2d45-469e-a76a-4930b73718fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:32.188888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-65a608c2-2d45-469e-a76a-4930b73718fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:32.190619 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1350/2686] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:41:32 2022] DELETE /identity/v3/projects/d7480570ee8b45ac809e2a4aadf2b779 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:32.195634 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a6dde-4130-46e7-adac-6a53c4f1edf8 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:32.196294 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a6dde-4130-46e7-adac-6a53c4f1edf8 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:32.196680 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a6dde-4130-46e7-adac-6a53c4f1edf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:32.215215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-362a6dde-4130-46e7-adac-6a53c4f1edf8 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:32.216555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-362a6dde-4130-46e7-adac-6a53c4f1edf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:32.230448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:32.290654 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1337/2687] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:41:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:41:32.304131 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6134de-7e02-4ee3-80f3-77cb00c2d770 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:32.304131 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6134de-7e02-4ee3-80f3-77cb00c2d770 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:32.304131 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6134de-7e02-4ee3-80f3-77cb00c2d770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:32.323591 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-1d6134de-7e02-4ee3-80f3-77cb00c2d770 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:32.325325 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1d6134de-7e02-4ee3-80f3-77cb00c2d770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:32.335092 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1351/2688] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:41:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:41:32.340884 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a6dde-4130-46e7-adac-6a53c4f1edf8 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:32.353237 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a6dde-4130-46e7-adac-6a53c4f1edf8 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:32.372602 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da5cce-ff5a-4b31-8c95-88834f00820f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:32.373317 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da5cce-ff5a-4b31-8c95-88834f00820f admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:32.373615 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da5cce-ff5a-4b31-8c95-88834f00820f admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:32.373901 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da5cce-ff5a-4b31-8c95-88834f00820f admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:32.374671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da5cce-ff5a-4b31-8c95-88834f00820f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:41:32.374973 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da5cce-ff5a-4b31-8c95-88834f00820f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:32.376814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da5cce-ff5a-4b31-8c95-88834f00820f admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:32.395200 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1338/2689] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:41:32.402073 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6134de-7e02-4ee3-80f3-77cb00c2d770 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:32.412977 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6134de-7e02-4ee3-80f3-77cb00c2d770 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:32.419972 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd8561-5d63-41eb-a478-27139aab69ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:32.420860 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd8561-5d63-41eb-a478-27139aab69ce admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:32.421320 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd8561-5d63-41eb-a478-27139aab69ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:32.421721 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fd8561-5d63-41eb-a478-27139aab69ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:32.422489 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fd8561-5d63-41eb-a478-27139aab69ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:32.424968 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fd8561-5d63-41eb-a478-27139aab69ce admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:32.438915 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:32.465831 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1352/2690] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:41:32 2022] POST /identity/v3/projects => generated 403 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:41:32.475716 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da5cce-ff5a-4b31-8c95-88834f00820f admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:32.490520 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da5cce-ff5a-4b31-8c95-88834f00820f admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:32.571505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6743bd6-a755-4afa-8078-64dd6eb9a85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:32.572211 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6743bd6-a755-4afa-8078-64dd6eb9a85f admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:32.572506 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6743bd6-a755-4afa-8078-64dd6eb9a85f admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:32.572798 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6743bd6-a755-4afa-8078-64dd6eb9a85f admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:32.573515 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6743bd6-a755-4afa-8078-64dd6eb9a85f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:32.575452 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6743bd6-a755-4afa-8078-64dd6eb9a85f admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:32.596803 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1339/2691] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:41:32 2022] POST /identity/v3/users => generated 302 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:41:32.600980 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd8561-5d63-41eb-a478-27139aab69ce admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:32.611801 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fd8561-5d63-41eb-a478-27139aab69ce admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:32.616340 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea131e0b-99cc-497c-af4d-dd0d76560346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:32.617011 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea131e0b-99cc-497c-af4d-dd0d76560346 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:32.617278 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea131e0b-99cc-497c-af4d-dd0d76560346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:32.617557 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea131e0b-99cc-497c-af4d-dd0d76560346 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:32.618275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea131e0b-99cc-497c-af4d-dd0d76560346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:32.619846 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea131e0b-99cc-497c-af4d-dd0d76560346 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:32.630110 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1353/2692] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:41:32.634360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6743bd6-a755-4afa-8078-64dd6eb9a85f admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:32.644373 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6743bd6-a755-4afa-8078-64dd6eb9a85f admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:32.649532 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe840e60-020a-499e-8bb0-43bb2cbafceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:32.650192 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe840e60-020a-499e-8bb0-43bb2cbafceb admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:32.650484 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe840e60-020a-499e-8bb0-43bb2cbafceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:32.650768 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe840e60-020a-499e-8bb0-43bb2cbafceb admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:32.651497 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe840e60-020a-499e-8bb0-43bb2cbafceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:32.653145 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe840e60-020a-499e-8bb0-43bb2cbafceb admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:32.663093 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1340/2693] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:41:32.667985 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea131e0b-99cc-497c-af4d-dd0d76560346 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:32.677838 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea131e0b-99cc-497c-af4d-dd0d76560346 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:32.682992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c919aeac-2c0a-4544-b606-76342dcf2096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:32.683906 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919aeac-2c0a-4544-b606-76342dcf2096 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:32.684201 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919aeac-2c0a-4544-b606-76342dcf2096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:32.686394 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919aeac-2c0a-4544-b606-76342dcf2096 admin admin] PATH_INFO: `/v3/projects/dd0bed247e6e4f3a9d315a791a9fff93/users/6304c27d783945cf8f9b0ec99faac7b2/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:32.700033 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c919aeac-2c0a-4544-b606-76342dcf2096 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd0bed247e6e4f3a9d315a791a9fff93, user_id=6304c27d783945cf8f9b0ec99faac7b2, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:32.702774 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c919aeac-2c0a-4544-b606-76342dcf2096 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:32.716594 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1354/2694] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:41:32 2022] PUT /identity/v3/projects/dd0bed247e6e4f3a9d315a791a9fff93/users/6304c27d783945cf8f9b0ec99faac7b2/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:32.728249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33f191-2f6c-4f49-8eb5-5c11254ace03 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:32.728552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33f191-2f6c-4f49-8eb5-5c11254ace03 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:32.728865 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33f191-2f6c-4f49-8eb5-5c11254ace03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:32.755130 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-5b33f191-2f6c-4f49-8eb5-5c11254ace03 None None] MFA Rules not processed for user `6304c27d783945cf8f9b0ec99faac7b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:32.756370 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-5b33f191-2f6c-4f49-8eb5-5c11254ace03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:32.828679 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1341/2695] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:41:32 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:41:32.833360 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c919aeac-2c0a-4544-b606-76342dcf2096 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:32.895676 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c919aeac-2c0a-4544-b606-76342dcf2096 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:32.901007 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07afcb-dc73-4aca-a5ab-c5daa5db7e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:32.901663 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07afcb-dc73-4aca-a5ab-c5daa5db7e19 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:32.901950 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07afcb-dc73-4aca-a5ab-c5daa5db7e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:32.902238 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07afcb-dc73-4aca-a5ab-c5daa5db7e19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:32.903367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07afcb-dc73-4aca-a5ab-c5daa5db7e19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:32.905929 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e07afcb-dc73-4aca-a5ab-c5daa5db7e19 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:32.930095 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1355/2696] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:41:32 2022] POST /identity/v3/projects => generated 403 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:41:32.934278 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33f191-2f6c-4f49-8eb5-5c11254ace03 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:32.945636 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33f191-2f6c-4f49-8eb5-5c11254ace03 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:32.954898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:33.004066 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4cc27-127f-4307-9bb4-4e042eaf2046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:33.004830 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4cc27-127f-4307-9bb4-4e042eaf2046 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:33.005118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4cc27-127f-4307-9bb4-4e042eaf2046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:33.005396 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4cc27-127f-4307-9bb4-4e042eaf2046 admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:33.006078 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4cc27-127f-4307-9bb4-4e042eaf2046 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:33.008128 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4cc27-127f-4307-9bb4-4e042eaf2046 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:33.030496 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1342/2697] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:41:32 2022] POST /identity/v3/users => generated 308 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:41:33.034478 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07afcb-dc73-4aca-a5ab-c5daa5db7e19 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:33.045809 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07afcb-dc73-4aca-a5ab-c5daa5db7e19 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:33.050406 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b289a-1cae-48ec-94d8-a9ab469b71ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:33.051071 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b289a-1cae-48ec-94d8-a9ab469b71ed admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:33.051360 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b289a-1cae-48ec-94d8-a9ab469b71ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:33.051641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b289a-1cae-48ec-94d8-a9ab469b71ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:33.052403 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b289a-1cae-48ec-94d8-a9ab469b71ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:33.054013 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b289a-1cae-48ec-94d8-a9ab469b71ed admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:33.065783 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1356/2698] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:41:33.070089 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4cc27-127f-4307-9bb4-4e042eaf2046 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:33.080853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4cc27-127f-4307-9bb4-4e042eaf2046 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:33.085428 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00619f49-3b31-471c-974d-69cc087e5ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:33.086964 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00619f49-3b31-471c-974d-69cc087e5ab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:33.087249 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00619f49-3b31-471c-974d-69cc087e5ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:33.087526 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00619f49-3b31-471c-974d-69cc087e5ab5 admin admin] PATH_INFO: `/v3/projects/dba8f09e9597451f8be07c2e4e142d0e/users/05230ed7f9634f24a46201444802caae/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:33.102777 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00619f49-3b31-471c-974d-69cc087e5ab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dba8f09e9597451f8be07c2e4e142d0e, user_id=05230ed7f9634f24a46201444802caae, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:33.105508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00619f49-3b31-471c-974d-69cc087e5ab5 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:33.118331 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1343/2699] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:41:33 2022] PUT /identity/v3/projects/dba8f09e9597451f8be07c2e4e142d0e/users/05230ed7f9634f24a46201444802caae/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:33.122227 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b289a-1cae-48ec-94d8-a9ab469b71ed admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:33.132038 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b289a-1cae-48ec-94d8-a9ab469b71ed admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:33.188976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6bf265-35b8-45f1-92fd-2ea236b319e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:33.189638 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6bf265-35b8-45f1-92fd-2ea236b319e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:33.189919 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6bf265-35b8-45f1-92fd-2ea236b319e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:33.190194 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6bf265-35b8-45f1-92fd-2ea236b319e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:33.190904 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6bf265-35b8-45f1-92fd-2ea236b319e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:33.192507 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6bf265-35b8-45f1-92fd-2ea236b319e3 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:33.202569 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1357/2700] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:41:33.206981 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00619f49-3b31-471c-974d-69cc087e5ab5 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:33.218001 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00619f49-3b31-471c-974d-69cc087e5ab5 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:33.222555 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12d53d-6a73-4796-ad7d-eb137533a7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:33.223429 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12d53d-6a73-4796-ad7d-eb137533a7c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:33.223710 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12d53d-6a73-4796-ad7d-eb137533a7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:33.223987 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12d53d-6a73-4796-ad7d-eb137533a7c4 admin admin] PATH_INFO: `/v3/projects/dba8f09e9597451f8be07c2e4e142d0e/users/05230ed7f9634f24a46201444802caae/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:33.225951 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12d53d-6a73-4796-ad7d-eb137533a7c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dba8f09e9597451f8be07c2e4e142d0e, user_id=05230ed7f9634f24a46201444802caae, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:33.229905 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12d53d-6a73-4796-ad7d-eb137533a7c4 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:33.239986 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1344/2701] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:41:33 2022] PUT /identity/v3/projects/dba8f09e9597451f8be07c2e4e142d0e/users/05230ed7f9634f24a46201444802caae/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:33.250393 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365999a0-3050-473b-aa18-14eed69efae8 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:33.250771 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365999a0-3050-473b-aa18-14eed69efae8 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:33.251120 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365999a0-3050-473b-aa18-14eed69efae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:33.277682 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-365999a0-3050-473b-aa18-14eed69efae8 None None] MFA Rules not processed for user `05230ed7f9634f24a46201444802caae`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:33.278888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-365999a0-3050-473b-aa18-14eed69efae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:33.293163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:33.348634 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1358/2702] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:41:33 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:41:33.360139 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12d53d-6a73-4796-ad7d-eb137533a7c4 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:33.362296 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-ef12d53d-6a73-4796-ad7d-eb137533a7c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:33.423898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12d53d-6a73-4796-ad7d-eb137533a7c4 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:33.427529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b188511b-e8d2-4470-8ae5-086d5a79c2ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:33.428201 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188511b-e8d2-4470-8ae5-086d5a79c2ad service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:33.428502 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188511b-e8d2-4470-8ae5-086d5a79c2ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:33.428788 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188511b-e8d2-4470-8ae5-086d5a79c2ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:33.438059 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b188511b-e8d2-4470-8ae5-086d5a79c2ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:33.439506 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b188511b-e8d2-4470-8ae5-086d5a79c2ad service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:33.453305 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1345/2703] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:41:33 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:41:33.482702 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee372333-1aa1-4298-84b7-6f3fd6c3cbc0 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:33.483257 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee372333-1aa1-4298-84b7-6f3fd6c3cbc0 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:33.483538 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee372333-1aa1-4298-84b7-6f3fd6c3cbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:33.505470 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-ee372333-1aa1-4298-84b7-6f3fd6c3cbc0 None None] MFA Rules not processed for user `733f8296ec3d4795a1140c1e1cb6eb95`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:33.506675 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ee372333-1aa1-4298-84b7-6f3fd6c3cbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:33.573048 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1359/2704] 10.222.0.16 () {58 vars in 967 bytes} [Mon May 23 12:41:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:41:33.587077 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b188511b-e8d2-4470-8ae5-086d5a79c2ad service cinder] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:33.588576 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-b188511b-e8d2-4470-8ae5-086d5a79c2ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:33.599096 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee372333-1aa1-4298-84b7-6f3fd6c3cbc0 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:33.662264 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b188511b-e8d2-4470-8ae5-086d5a79c2ad service cinder] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:33.668816 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee372333-1aa1-4298-84b7-6f3fd6c3cbc0 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:33.669435 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe8c06-cddf-4817-acea-d9647ec3abd6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71fb9a79ac494635804b93d4a0048205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:33.670118 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe8c06-cddf-4817-acea-d9647ec3abd6 service swift] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:33.670456 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe8c06-cddf-4817-acea-d9647ec3abd6 service swift] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:33.670686 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe8c06-cddf-4817-acea-d9647ec3abd6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:33.673673 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863a747-06db-43ba-97bd-1d2b89a0dd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:33.674547 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863a747-06db-43ba-97bd-1d2b89a0dd93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:33.674835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863a747-06db-43ba-97bd-1d2b89a0dd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:33.675114 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6863a747-06db-43ba-97bd-1d2b89a0dd93 admin admin] PATH_INFO: `/v3/users/6304c27d783945cf8f9b0ec99faac7b2` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:33.676331 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6863a747-06db-43ba-97bd-1d2b89a0dd93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6304c27d783945cf8f9b0ec99faac7b2)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:33.678252 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6863a747-06db-43ba-97bd-1d2b89a0dd93 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:33.680402 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fe8c06-cddf-4817-acea-d9647ec3abd6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:33.681769 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fe8c06-cddf-4817-acea-d9647ec3abd6 service swift] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:33.685490 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:33.690856 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1346/2705] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 12:41:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 12:41:33.729497 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6863a747-06db-43ba-97bd-1d2b89a0dd93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6304c27d783945cf8f9b0ec99faac7b2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:33.737253 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:33.743391 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6863a747-06db-43ba-97bd-1d2b89a0dd93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6304c27d783945cf8f9b0ec99faac7b2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:33.751957 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6863a747-06db-43ba-97bd-1d2b89a0dd93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6304c27d783945cf8f9b0ec99faac7b2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:33.752826 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6863a747-06db-43ba-97bd-1d2b89a0dd93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6304c27d783945cf8f9b0ec99faac7b2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:33.753433 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-6863a747-06db-43ba-97bd-1d2b89a0dd93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6304c27d783945cf8f9b0ec99faac7b2'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:33.767124 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1360/2706] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:41:33 2022] DELETE /identity/v3/users/6304c27d783945cf8f9b0ec99faac7b2 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:41:33.771721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ba03c-be7f-4f54-84ac-f24d3b39908d None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:33.772006 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ba03c-be7f-4f54-84ac-f24d3b39908d None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:33.772331 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ba03c-be7f-4f54-84ac-f24d3b39908d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:33.796525 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-5b7ba03c-be7f-4f54-84ac-f24d3b39908d None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:33.798004 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-5b7ba03c-be7f-4f54-84ac-f24d3b39908d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:33.874787 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1347/2707] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:41:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:41:33.884890 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863a747-06db-43ba-97bd-1d2b89a0dd93 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:33.886503 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-6863a747-06db-43ba-97bd-1d2b89a0dd93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:33.951695 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6863a747-06db-43ba-97bd-1d2b89a0dd93 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:33.956264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca45727-ea9b-4df2-b9fb-d138186b2349 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:33.956932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca45727-ea9b-4df2-b9fb-d138186b2349 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:33.957230 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca45727-ea9b-4df2-b9fb-d138186b2349 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:33.957546 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca45727-ea9b-4df2-b9fb-d138186b2349 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:33.967932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca45727-ea9b-4df2-b9fb-d138186b2349 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:33.969393 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca45727-ea9b-4df2-b9fb-d138186b2349 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:33.980299 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1361/2708] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:41:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:41:34.271369 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ba03c-be7f-4f54-84ac-f24d3b39908d None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:34.273369 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-5b7ba03c-be7f-4f54-84ac-f24d3b39908d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:34.291222 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ba03c-be7f-4f54-84ac-f24d3b39908d None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:34.297284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d3da3-c828-479d-8d42-bf1fdf8a7c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:34.298105 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d3da3-c828-479d-8d42-bf1fdf8a7c4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:34.298408 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d3da3-c828-479d-8d42-bf1fdf8a7c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:34.298626 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d3da3-c828-479d-8d42-bf1fdf8a7c4e admin admin] PATH_INFO: `/v3/projects/dd0bed247e6e4f3a9d315a791a9fff93` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:34.300238 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d3da3-c828-479d-8d42-bf1fdf8a7c4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd0bed247e6e4f3a9d315a791a9fff93)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:34.302162 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d3da3-c828-479d-8d42-bf1fdf8a7c4e admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:34.350290 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-756d3da3-c828-479d-8d42-bf1fdf8a7c4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd0bed247e6e4f3a9d315a791a9fff93', 'request_id': 'req-756d3da3-c828-479d-8d42-bf1fdf8a7c4e'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:34.358746 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-756d3da3-c828-479d-8d42-bf1fdf8a7c4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd0bed247e6e4f3a9d315a791a9fff93', 'request_id': 'req-756d3da3-c828-479d-8d42-bf1fdf8a7c4e'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:34.365236 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-756d3da3-c828-479d-8d42-bf1fdf8a7c4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd0bed247e6e4f3a9d315a791a9fff93', 'request_id': 'req-756d3da3-c828-479d-8d42-bf1fdf8a7c4e'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:34.376607 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-756d3da3-c828-479d-8d42-bf1fdf8a7c4e admin admin] The token cache is being invalidate because project dd0bed247e6e4f3a9d315a791a9fff93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:41:34.376824 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-756d3da3-c828-479d-8d42-bf1fdf8a7c4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:34.377457 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-756d3da3-c828-479d-8d42-bf1fdf8a7c4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:34.379015 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1348/2709] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:41:34 2022] DELETE /identity/v3/projects/dd0bed247e6e4f3a9d315a791a9fff93 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:34.383385 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca45727-ea9b-4df2-b9fb-d138186b2349 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:34.392976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca45727-ea9b-4df2-b9fb-d138186b2349 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:34.448897 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512e3539-caab-4db7-8245-ce68473c9ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:34.449754 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e3539-caab-4db7-8245-ce68473c9ad7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:34.450063 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e3539-caab-4db7-8245-ce68473c9ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:34.450340 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e3539-caab-4db7-8245-ce68473c9ad7 admin admin] PATH_INFO: `/v3/users/05230ed7f9634f24a46201444802caae` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:34.451514 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512e3539-caab-4db7-8245-ce68473c9ad7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05230ed7f9634f24a46201444802caae)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:34.453536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512e3539-caab-4db7-8245-ce68473c9ad7 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:34.510183 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-512e3539-caab-4db7-8245-ce68473c9ad7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05230ed7f9634f24a46201444802caae'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:34.521916 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-512e3539-caab-4db7-8245-ce68473c9ad7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05230ed7f9634f24a46201444802caae'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:34.529846 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-512e3539-caab-4db7-8245-ce68473c9ad7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05230ed7f9634f24a46201444802caae'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:34.530463 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-512e3539-caab-4db7-8245-ce68473c9ad7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05230ed7f9634f24a46201444802caae'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:34.531093 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-512e3539-caab-4db7-8245-ce68473c9ad7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05230ed7f9634f24a46201444802caae'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:34.539432 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:34.545400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1362/2710] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:41:34 2022] DELETE /identity/v3/users/05230ed7f9634f24a46201444802caae => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:34.815774 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d3da3-c828-479d-8d42-bf1fdf8a7c4e admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:34.817318 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-756d3da3-c828-479d-8d42-bf1fdf8a7c4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:34.833155 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d3da3-c828-479d-8d42-bf1fdf8a7c4e admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:34.892414 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084081bf-2f70-454c-9844-d1d9b2228293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:34.893199 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084081bf-2f70-454c-9844-d1d9b2228293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:34.893520 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084081bf-2f70-454c-9844-d1d9b2228293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:34.893800 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084081bf-2f70-454c-9844-d1d9b2228293 admin admin] PATH_INFO: `/v3/projects/dba8f09e9597451f8be07c2e4e142d0e` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:34.894982 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084081bf-2f70-454c-9844-d1d9b2228293 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dba8f09e9597451f8be07c2e4e142d0e)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:34.896950 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084081bf-2f70-454c-9844-d1d9b2228293 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:34.922682 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:34.945391 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-084081bf-2f70-454c-9844-d1d9b2228293 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dba8f09e9597451f8be07c2e4e142d0e', 'request_id': 'req-084081bf-2f70-454c-9844-d1d9b2228293'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:34.952159 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-084081bf-2f70-454c-9844-d1d9b2228293 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dba8f09e9597451f8be07c2e4e142d0e', 'request_id': 'req-084081bf-2f70-454c-9844-d1d9b2228293'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:34.958778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-084081bf-2f70-454c-9844-d1d9b2228293 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dba8f09e9597451f8be07c2e4e142d0e', 'request_id': 'req-084081bf-2f70-454c-9844-d1d9b2228293'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:34.969750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-084081bf-2f70-454c-9844-d1d9b2228293 admin admin] The token cache is being invalidate because project dba8f09e9597451f8be07c2e4e142d0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:41:34.970043 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-084081bf-2f70-454c-9844-d1d9b2228293 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:34.970683 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-084081bf-2f70-454c-9844-d1d9b2228293 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:34.972240 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1349/2711] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:41:34 2022] DELETE /identity/v3/projects/dba8f09e9597451f8be07c2e4e142d0e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:34.977359 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba1140-a696-4b04-ba7a-90b66540d880 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:34.977648 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba1140-a696-4b04-ba7a-90b66540d880 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:34.977933 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba1140-a696-4b04-ba7a-90b66540d880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:34.995991 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-ccba1140-a696-4b04-ba7a-90b66540d880 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:34.997193 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ccba1140-a696-4b04-ba7a-90b66540d880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:35.058807 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1363/2712] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:41:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:41:35.066462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e776464a-0c9c-4718-96b7-9b9e0abf530a None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:35.066755 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e776464a-0c9c-4718-96b7-9b9e0abf530a None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:35.067042 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e776464a-0c9c-4718-96b7-9b9e0abf530a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:35.088375 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-e776464a-0c9c-4718-96b7-9b9e0abf530a None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:35.089576 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-e776464a-0c9c-4718-96b7-9b9e0abf530a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:35.097460 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1350/2713] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:41:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:41:35.102376 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccba1140-a696-4b04-ba7a-90b66540d880 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:35.112260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccba1140-a696-4b04-ba7a-90b66540d880 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:35.117594 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5995f7df-1752-49c3-adf9-fc1f4d700bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:35.118247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995f7df-1752-49c3-adf9-fc1f4d700bd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:35.118532 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995f7df-1752-49c3-adf9-fc1f4d700bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:35.118804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995f7df-1752-49c3-adf9-fc1f4d700bd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:35.119525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5995f7df-1752-49c3-adf9-fc1f4d700bd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:41:35.119829 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5995f7df-1752-49c3-adf9-fc1f4d700bd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:35.121444 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5995f7df-1752-49c3-adf9-fc1f4d700bd5 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:35.138314 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1364/2714] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:41:35.142722 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e776464a-0c9c-4718-96b7-9b9e0abf530a None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:35.152649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e776464a-0c9c-4718-96b7-9b9e0abf530a None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:35.157457 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e0975-f825-4e72-9608-92ce91041abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:35.158111 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e0975-f825-4e72-9608-92ce91041abb admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:35.158397 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e0975-f825-4e72-9608-92ce91041abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:35.158676 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e0975-f825-4e72-9608-92ce91041abb admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:35.159413 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e0975-f825-4e72-9608-92ce91041abb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:35.161352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e0975-f825-4e72-9608-92ce91041abb admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:35.191340 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1351/2715] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:41:35 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:41:35.192573 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5995f7df-1752-49c3-adf9-fc1f4d700bd5 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:35.203173 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5995f7df-1752-49c3-adf9-fc1f4d700bd5 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:35.261243 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4244dcd-88e9-4eba-bee2-ace39385ca7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:35.261930 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4244dcd-88e9-4eba-bee2-ace39385ca7c admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:35.262216 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4244dcd-88e9-4eba-bee2-ace39385ca7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:35.262497 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4244dcd-88e9-4eba-bee2-ace39385ca7c admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:35.263225 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4244dcd-88e9-4eba-bee2-ace39385ca7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:35.265188 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4244dcd-88e9-4eba-bee2-ace39385ca7c admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:35.286479 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1365/2716] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:41:35 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:41:35.290610 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e0975-f825-4e72-9608-92ce91041abb admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:35.301704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e0975-f825-4e72-9608-92ce91041abb admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:35.307364 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd956e93-3f74-4480-8fa2-1ad7a165db95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:35.308066 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd956e93-3f74-4480-8fa2-1ad7a165db95 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:35.308405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd956e93-3f74-4480-8fa2-1ad7a165db95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:35.308692 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd956e93-3f74-4480-8fa2-1ad7a165db95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:35.309429 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd956e93-3f74-4480-8fa2-1ad7a165db95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:35.311049 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd956e93-3f74-4480-8fa2-1ad7a165db95 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:35.322778 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1352/2717] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:41:35.326597 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4244dcd-88e9-4eba-bee2-ace39385ca7c admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:35.337321 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4244dcd-88e9-4eba-bee2-ace39385ca7c admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:35.342075 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02348627-ab44-47fa-957b-8e4a3fc5c336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:35.342756 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02348627-ab44-47fa-957b-8e4a3fc5c336 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:35.343138 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02348627-ab44-47fa-957b-8e4a3fc5c336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:35.343852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02348627-ab44-47fa-957b-8e4a3fc5c336 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:35.344051 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02348627-ab44-47fa-957b-8e4a3fc5c336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:35.345699 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02348627-ab44-47fa-957b-8e4a3fc5c336 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:35.359916 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1366/2718] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:41:35.364024 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd956e93-3f74-4480-8fa2-1ad7a165db95 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:35.374479 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd956e93-3f74-4480-8fa2-1ad7a165db95 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:35.379202 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e9373-9859-48e8-8b09-77c1f570d76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:35.380251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e9373-9859-48e8-8b09-77c1f570d76d admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:35.380607 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e9373-9859-48e8-8b09-77c1f570d76d admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:35.380959 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8e9373-9859-48e8-8b09-77c1f570d76d admin admin] PATH_INFO: `/v3/projects/44a9eff003944792a6a51c027004d8db/users/a458671a53ca436cbb10bb1f2d5cfc04/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:35.396089 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8e9373-9859-48e8-8b09-77c1f570d76d admin admin] RBAC: Authorizing `identity:create_grant(project_id=44a9eff003944792a6a51c027004d8db, user_id=a458671a53ca436cbb10bb1f2d5cfc04, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:35.399412 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8e9373-9859-48e8-8b09-77c1f570d76d admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:35.415153 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1353/2719] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:41:35 2022] PUT /identity/v3/projects/44a9eff003944792a6a51c027004d8db/users/a458671a53ca436cbb10bb1f2d5cfc04/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:35.426300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41af8b-f802-4a0a-ae48-8772a249bf65 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:35.426796 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41af8b-f802-4a0a-ae48-8772a249bf65 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:35.426921 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf41af8b-f802-4a0a-ae48-8772a249bf65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:35.454815 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-cf41af8b-f802-4a0a-ae48-8772a249bf65 None None] MFA Rules not processed for user `a458671a53ca436cbb10bb1f2d5cfc04`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:35.456025 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-cf41af8b-f802-4a0a-ae48-8772a249bf65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:35.513313 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:35.526108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1367/2720] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:41:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:41:35.530301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e9373-9859-48e8-8b09-77c1f570d76d admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:35.597911 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8e9373-9859-48e8-8b09-77c1f570d76d admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:35.603848 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d68af-f77c-4b5b-986b-2d3354b39453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:35.605337 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d68af-f77c-4b5b-986b-2d3354b39453 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:35.605724 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d68af-f77c-4b5b-986b-2d3354b39453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:35.606146 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d68af-f77c-4b5b-986b-2d3354b39453 admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:35.607073 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01d68af-f77c-4b5b-986b-2d3354b39453 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:35.609184 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01d68af-f77c-4b5b-986b-2d3354b39453 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:35.639253 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1354/2721] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:41:35 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:41:35.643560 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41af8b-f802-4a0a-ae48-8772a249bf65 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:35.658964 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf41af8b-f802-4a0a-ae48-8772a249bf65 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:35.665608 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d68af-f77c-4b5b-986b-2d3354b39453 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:35.668409 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-f01d68af-f77c-4b5b-986b-2d3354b39453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:35.714509 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:35.721939 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a940082f-876b-4cc9-b58d-470c5be9f008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:35.722619 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a940082f-876b-4cc9-b58d-470c5be9f008 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:35.722909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a940082f-876b-4cc9-b58d-470c5be9f008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:35.723192 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a940082f-876b-4cc9-b58d-470c5be9f008 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:35.724162 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a940082f-876b-4cc9-b58d-470c5be9f008 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:35.726426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a940082f-876b-4cc9-b58d-470c5be9f008 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:35.747069 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d68af-f77c-4b5b-986b-2d3354b39453 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:35.748840 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1368/2722] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:41:35 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:41:35.752767 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a940082f-876b-4cc9-b58d-470c5be9f008 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:35.754720 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c267c-a783-438d-b373-22c8f4bacaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:35.755599 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c267c-a783-438d-b373-22c8f4bacaca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:35.755885 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c267c-a783-438d-b373-22c8f4bacaca admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:35.756165 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c267c-a783-438d-b373-22c8f4bacaca admin admin] PATH_INFO: `/v3/users/be8b5c7b8f48419ebcae6a1913661afa` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:35.757974 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c267c-a783-438d-b373-22c8f4bacaca admin admin] RBAC: Authorizing `identity:delete_user(user_id=be8b5c7b8f48419ebcae6a1913661afa)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:35.759985 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c267c-a783-438d-b373-22c8f4bacaca admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:35.764604 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a940082f-876b-4cc9-b58d-470c5be9f008 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:35.769250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfecff-44b0-4e3f-a4c3-8fe0cc30d8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:35.769909 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfecff-44b0-4e3f-a4c3-8fe0cc30d8cc admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:35.770192 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfecff-44b0-4e3f-a4c3-8fe0cc30d8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:35.770480 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfecff-44b0-4e3f-a4c3-8fe0cc30d8cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:35.771226 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfecff-44b0-4e3f-a4c3-8fe0cc30d8cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:35.772826 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfecff-44b0-4e3f-a4c3-8fe0cc30d8cc admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:35.783459 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1369/2723] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:41:35.787538 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfecff-44b0-4e3f-a4c3-8fe0cc30d8cc admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:35.798070 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfecff-44b0-4e3f-a4c3-8fe0cc30d8cc admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:35.802692 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef57164-a647-43c3-b5ab-bb88ef4a70f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:35.803572 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef57164-a647-43c3-b5ab-bb88ef4a70f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:35.803863 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef57164-a647-43c3-b5ab-bb88ef4a70f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:35.804659 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef57164-a647-43c3-b5ab-bb88ef4a70f1 admin admin] PATH_INFO: `/v3/projects/56c7c4debeef4cb4be711bf1b9629006/users/7bb2de5b0bb2441ab0c52aa32400d265/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:35.812089 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-5c3c267c-a783-438d-b373-22c8f4bacaca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be8b5c7b8f48419ebcae6a1913661afa'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:35.812795 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-5c3c267c-a783-438d-b373-22c8f4bacaca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be8b5c7b8f48419ebcae6a1913661afa'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:35.820897 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-5c3c267c-a783-438d-b373-22c8f4bacaca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be8b5c7b8f48419ebcae6a1913661afa'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:35.821037 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef57164-a647-43c3-b5ab-bb88ef4a70f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56c7c4debeef4cb4be711bf1b9629006, user_id=7bb2de5b0bb2441ab0c52aa32400d265, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:35.823539 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef57164-a647-43c3-b5ab-bb88ef4a70f1 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:35.833527 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-5c3c267c-a783-438d-b373-22c8f4bacaca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be8b5c7b8f48419ebcae6a1913661afa'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:35.837858 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1370/2724] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:41:35 2022] PUT /identity/v3/projects/56c7c4debeef4cb4be711bf1b9629006/users/7bb2de5b0bb2441ab0c52aa32400d265/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:41:35.841926 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef57164-a647-43c3-b5ab-bb88ef4a70f1 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:35.842762 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-5c3c267c-a783-438d-b373-22c8f4bacaca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be8b5c7b8f48419ebcae6a1913661afa'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:35.849319 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1355/2725] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:41:35 2022] DELETE /identity/v3/users/be8b5c7b8f48419ebcae6a1913661afa => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:35.913156 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef57164-a647-43c3-b5ab-bb88ef4a70f1 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:35.968306 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:35.976318 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031559a-a83e-492a-99a0-2b8a25544f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:35.977210 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031559a-a83e-492a-99a0-2b8a25544f1c admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:35.977733 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031559a-a83e-492a-99a0-2b8a25544f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:35.978118 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031559a-a83e-492a-99a0-2b8a25544f1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:35.978953 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8031559a-a83e-492a-99a0-2b8a25544f1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:35.980747 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8031559a-a83e-492a-99a0-2b8a25544f1c admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:35.995355 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1371/2726] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:41:35.996789 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c267c-a783-438d-b373-22c8f4bacaca admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:35.998287 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-5c3c267c-a783-438d-b373-22c8f4bacaca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:36.011838 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c267c-a783-438d-b373-22c8f4bacaca admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:36.016898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b03aa3d-8eac-4ad4-84bb-7661896c736e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:36.017868 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03aa3d-8eac-4ad4-84bb-7661896c736e admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:36.018248 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03aa3d-8eac-4ad4-84bb-7661896c736e admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:36.024048 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03aa3d-8eac-4ad4-84bb-7661896c736e admin admin] PATH_INFO: `/v3/projects/56c7c4debeef4cb4be711bf1b9629006/users/7bb2de5b0bb2441ab0c52aa32400d265/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:36.024048 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03aa3d-8eac-4ad4-84bb-7661896c736e admin admin] RBAC: Authorizing `identity:create_grant(project_id=56c7c4debeef4cb4be711bf1b9629006, user_id=7bb2de5b0bb2441ab0c52aa32400d265, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:36.024323 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b03aa3d-8eac-4ad4-84bb-7661896c736e admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:36.033214 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031559a-a83e-492a-99a0-2b8a25544f1c admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:36.035083 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-8031559a-a83e-492a-99a0-2b8a25544f1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:36.042325 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1356/2727] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:41:35 2022] PUT /identity/v3/projects/56c7c4debeef4cb4be711bf1b9629006/users/7bb2de5b0bb2441ab0c52aa32400d265/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:36.057240 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6764a-fc14-472c-bc54-720561fba87d None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:36.057449 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031559a-a83e-492a-99a0-2b8a25544f1c admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:36.058221 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6764a-fc14-472c-bc54-720561fba87d None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:36.058584 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6764a-fc14-472c-bc54-720561fba87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:36.088766 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-dfd6764a-fc14-472c-bc54-720561fba87d None None] MFA Rules not processed for user `7bb2de5b0bb2441ab0c52aa32400d265`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:36.090171 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-dfd6764a-fc14-472c-bc54-720561fba87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:36.123231 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde2cde-e94b-41cd-be59-710be1f52013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:36.124200 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde2cde-e94b-41cd-be59-710be1f52013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:36.124415 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde2cde-e94b-41cd-be59-710be1f52013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:36.124691 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde2cde-e94b-41cd-be59-710be1f52013 admin admin] PATH_INFO: `/v3/projects/c1f2700efc5d45119cc13a60bc0f25f0` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:36.125859 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde2cde-e94b-41cd-be59-710be1f52013 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1f2700efc5d45119cc13a60bc0f25f0)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:36.127880 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adde2cde-e94b-41cd-be59-710be1f52013 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:36.159498 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1357/2728] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:41:36 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:41:36.171382 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd6764a-fc14-472c-bc54-720561fba87d None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:36.172746 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-dfd6764a-fc14-472c-bc54-720561fba87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:36.177508 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-adde2cde-e94b-41cd-be59-710be1f52013 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1f2700efc5d45119cc13a60bc0f25f0', 'request_id': 'req-adde2cde-e94b-41cd-be59-710be1f52013'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:36.184768 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-adde2cde-e94b-41cd-be59-710be1f52013 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1f2700efc5d45119cc13a60bc0f25f0', 'request_id': 'req-adde2cde-e94b-41cd-be59-710be1f52013'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:36.191059 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-adde2cde-e94b-41cd-be59-710be1f52013 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1f2700efc5d45119cc13a60bc0f25f0', 'request_id': 'req-adde2cde-e94b-41cd-be59-710be1f52013'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:36.202400 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:36.204248 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-adde2cde-e94b-41cd-be59-710be1f52013 admin admin] The token cache is being invalidate because project c1f2700efc5d45119cc13a60bc0f25f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:41:36.204852 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-adde2cde-e94b-41cd-be59-710be1f52013 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:36.205436 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-adde2cde-e94b-41cd-be59-710be1f52013 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:36.206989 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1372/2729] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:41:36 2022] DELETE /identity/v3/projects/c1f2700efc5d45119cc13a60bc0f25f0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:41:36.211807 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff126d68-bed6-483a-91d8-1c6d5bc165db None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:36.212101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff126d68-bed6-483a-91d8-1c6d5bc165db None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:36.212415 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff126d68-bed6-483a-91d8-1c6d5bc165db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:36.232086 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-ff126d68-bed6-483a-91d8-1c6d5bc165db None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:36.233326 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-ff126d68-bed6-483a-91d8-1c6d5bc165db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:36.237548 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd6764a-fc14-472c-bc54-720561fba87d None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:36.292134 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60c45a-ef4b-445e-aa9f-841d08f750f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:36.292860 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60c45a-ef4b-445e-aa9f-841d08f750f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:36.293090 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60c45a-ef4b-445e-aa9f-841d08f750f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:36.293366 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60c45a-ef4b-445e-aa9f-841d08f750f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:36.295116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-6c60c45a-ef4b-445e-aa9f-841d08f750f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:36.295512 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1373/2730] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:41:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:41:36.302755 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb74ff-b92f-4a4b-8dc3-cc2a9aa47544 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:36.303032 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb74ff-b92f-4a4b-8dc3-cc2a9aa47544 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:36.303388 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb74ff-b92f-4a4b-8dc3-cc2a9aa47544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:36.324595 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-eecb74ff-b92f-4a4b-8dc3-cc2a9aa47544 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:36.326008 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-eecb74ff-b92f-4a4b-8dc3-cc2a9aa47544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:36.334411 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1374/2731] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:41:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:41:36.339129 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb74ff-b92f-4a4b-8dc3-cc2a9aa47544 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:36.348015 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb74ff-b92f-4a4b-8dc3-cc2a9aa47544 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:36.352956 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f94e8-c28c-49e5-9c6b-4a79682df118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:36.353597 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f94e8-c28c-49e5-9c6b-4a79682df118 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:36.353878 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f94e8-c28c-49e5-9c6b-4a79682df118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:36.354268 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f94e8-c28c-49e5-9c6b-4a79682df118 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:36.354992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64f94e8-c28c-49e5-9c6b-4a79682df118 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:41:36.355301 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64f94e8-c28c-49e5-9c6b-4a79682df118 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:36.356956 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64f94e8-c28c-49e5-9c6b-4a79682df118 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:36.362874 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60c45a-ef4b-445e-aa9f-841d08f750f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:36.364534 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60c45a-ef4b-445e-aa9f-841d08f750f0 service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:36.368521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1375/2732] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:41:36.373436 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f94e8-c28c-49e5-9c6b-4a79682df118 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:36.375942 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1358/2733] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:41:36 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:41:36.378869 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:36.386189 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f94e8-c28c-49e5-9c6b-4a79682df118 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:36.392259 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfec62c-a515-4153-8afc-259b5c4c20ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:36.393056 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfec62c-a515-4153-8afc-259b5c4c20ab admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:36.393343 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfec62c-a515-4153-8afc-259b5c4c20ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:36.393626 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfec62c-a515-4153-8afc-259b5c4c20ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:36.394308 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfec62c-a515-4153-8afc-259b5c4c20ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:36.396349 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfec62c-a515-4153-8afc-259b5c4c20ab admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:36.424636 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1376/2734] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:41:36 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:41:36.432260 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60c45a-ef4b-445e-aa9f-841d08f750f0 service cinder] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:36.446581 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60c45a-ef4b-445e-aa9f-841d08f750f0 service cinder] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:36.513244 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46714454-f989-4164-894c-886d4d50eed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:36.514821 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46714454-f989-4164-894c-886d4d50eed3 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:36.514821 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46714454-f989-4164-894c-886d4d50eed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:36.514821 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46714454-f989-4164-894c-886d4d50eed3 admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:36.515606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46714454-f989-4164-894c-886d4d50eed3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:36.517577 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46714454-f989-4164-894c-886d4d50eed3 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:36.543631 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1359/2735] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:41:36 2022] POST /identity/v3/users => generated 285 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:41:36.547704 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfec62c-a515-4153-8afc-259b5c4c20ab admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:36.559492 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfec62c-a515-4153-8afc-259b5c4c20ab admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:36.564533 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f3aba-da82-4f9d-a7ac-88b69a574f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:36.565265 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f3aba-da82-4f9d-a7ac-88b69a574f27 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:36.565466 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f3aba-da82-4f9d-a7ac-88b69a574f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:36.565742 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f3aba-da82-4f9d-a7ac-88b69a574f27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:36.566489 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51f3aba-da82-4f9d-a7ac-88b69a574f27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:36.568066 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51f3aba-da82-4f9d-a7ac-88b69a574f27 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:36.583986 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1377/2736] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:41:36.585986 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46714454-f989-4164-894c-886d4d50eed3 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:36.596580 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46714454-f989-4164-894c-886d4d50eed3 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:36.602428 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f2a919-8180-42e5-b0a9-3fdd3375f988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:36.603250 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2a919-8180-42e5-b0a9-3fdd3375f988 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:36.603634 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2a919-8180-42e5-b0a9-3fdd3375f988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:36.604010 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f2a919-8180-42e5-b0a9-3fdd3375f988 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:36.604897 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f2a919-8180-42e5-b0a9-3fdd3375f988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:36.606535 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f2a919-8180-42e5-b0a9-3fdd3375f988 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:36.618169 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1360/2737] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:41:36.621346 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f3aba-da82-4f9d-a7ac-88b69a574f27 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:36.631485 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f3aba-da82-4f9d-a7ac-88b69a574f27 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:36.636295 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506485c6-8240-4a9a-b92f-a820f504f880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:36.637184 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506485c6-8240-4a9a-b92f-a820f504f880 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:36.637468 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506485c6-8240-4a9a-b92f-a820f504f880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:36.637753 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506485c6-8240-4a9a-b92f-a820f504f880 admin admin] PATH_INFO: `/v3/projects/66d7a2d57eec445ca7f2d95a15f28b8b/users/b6f6c6a6728b4a9bb00d6f7eaae715d7/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:36.656296 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506485c6-8240-4a9a-b92f-a820f504f880 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66d7a2d57eec445ca7f2d95a15f28b8b, user_id=b6f6c6a6728b4a9bb00d6f7eaae715d7, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:36.659012 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506485c6-8240-4a9a-b92f-a820f504f880 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:36.674926 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1378/2738] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:41:36 2022] PUT /identity/v3/projects/66d7a2d57eec445ca7f2d95a15f28b8b/users/b6f6c6a6728b4a9bb00d6f7eaae715d7/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:36.684899 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0301d7-6ccb-45b5-823c-b5a94d26de24 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:36.685204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0301d7-6ccb-45b5-823c-b5a94d26de24 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:36.685488 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0301d7-6ccb-45b5-823c-b5a94d26de24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:36.704619 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:36.715964 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-ed0301d7-6ccb-45b5-823c-b5a94d26de24 None None] MFA Rules not processed for user `b6f6c6a6728b4a9bb00d6f7eaae715d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:36.717218 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-ed0301d7-6ccb-45b5-823c-b5a94d26de24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:36.753441 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506485c6-8240-4a9a-b92f-a820f504f880 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:36.755198 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-506485c6-8240-4a9a-b92f-a820f504f880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:36.786226 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1361/2739] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:41:36 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:41:36.797957 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0301d7-6ccb-45b5-823c-b5a94d26de24 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:36.799423 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-ed0301d7-6ccb-45b5-823c-b5a94d26de24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:36.829810 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506485c6-8240-4a9a-b92f-a820f504f880 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:36.834678 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185272a-aca8-473b-ab6a-b066a2a6a85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:36.836299 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185272a-aca8-473b-ab6a-b066a2a6a85f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:36.836804 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185272a-aca8-473b-ab6a-b066a2a6a85f admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:36.837150 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185272a-aca8-473b-ab6a-b066a2a6a85f admin admin] PATH_INFO: `/v3/users/a458671a53ca436cbb10bb1f2d5cfc04` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:36.838348 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2185272a-aca8-473b-ab6a-b066a2a6a85f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a458671a53ca436cbb10bb1f2d5cfc04)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:36.840347 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2185272a-aca8-473b-ab6a-b066a2a6a85f admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:36.860104 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0301d7-6ccb-45b5-823c-b5a94d26de24 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:36.865341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c34435-952b-4ef8-bd77-ab62af1e01ed service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2483a351f5e943ad8c872a98a4717b0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:36.866154 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c34435-952b-4ef8-bd77-ab62af1e01ed service glance] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:36.866623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c34435-952b-4ef8-bd77-ab62af1e01ed service glance] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:36.866943 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c34435-952b-4ef8-bd77-ab62af1e01ed service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:36.876849 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c34435-952b-4ef8-bd77-ab62af1e01ed service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:36.878286 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c34435-952b-4ef8-bd77-ab62af1e01ed service glance] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:36.888450 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1362/2740] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:41:36 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:41:36.889005 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2185272a-aca8-473b-ab6a-b066a2a6a85f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a458671a53ca436cbb10bb1f2d5cfc04'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:36.901693 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2185272a-aca8-473b-ab6a-b066a2a6a85f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a458671a53ca436cbb10bb1f2d5cfc04'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:36.910518 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2185272a-aca8-473b-ab6a-b066a2a6a85f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a458671a53ca436cbb10bb1f2d5cfc04'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:36.911195 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2185272a-aca8-473b-ab6a-b066a2a6a85f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a458671a53ca436cbb10bb1f2d5cfc04'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:36.911843 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-2185272a-aca8-473b-ab6a-b066a2a6a85f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a458671a53ca436cbb10bb1f2d5cfc04'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:36.925678 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1379/2741] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:41:36 2022] DELETE /identity/v3/users/a458671a53ca436cbb10bb1f2d5cfc04 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:41:36.929978 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55617ebb-c0e5-4a73-8632-c14c27892a5d None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:36.930495 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55617ebb-c0e5-4a73-8632-c14c27892a5d None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:36.930784 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55617ebb-c0e5-4a73-8632-c14c27892a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:36.948997 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-55617ebb-c0e5-4a73-8632-c14c27892a5d None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:36.950204 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-55617ebb-c0e5-4a73-8632-c14c27892a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:37.015894 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1363/2742] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:41:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:41:37.025683 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185272a-aca8-473b-ab6a-b066a2a6a85f admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:37.027140 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-2185272a-aca8-473b-ab6a-b066a2a6a85f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:37.059529 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:37.097178 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185272a-aca8-473b-ab6a-b066a2a6a85f admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:37.100863 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028af4f9-4040-4d78-bb6a-12a6a4ee4a78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:37.102415 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028af4f9-4040-4d78-bb6a-12a6a4ee4a78 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:37.102821 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028af4f9-4040-4d78-bb6a-12a6a4ee4a78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:37.103239 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028af4f9-4040-4d78-bb6a-12a6a4ee4a78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:37.113769 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028af4f9-4040-4d78-bb6a-12a6a4ee4a78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:37.115271 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028af4f9-4040-4d78-bb6a-12a6a4ee4a78 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:37.126151 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1380/2743] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:41:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:41:37.466568 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55617ebb-c0e5-4a73-8632-c14c27892a5d None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:37.468259 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-55617ebb-c0e5-4a73-8632-c14c27892a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:37.482710 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55617ebb-c0e5-4a73-8632-c14c27892a5d None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:37.488088 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6f542-7442-4be9-9d92-051d5fc39d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:37.489494 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6f542-7442-4be9-9d92-051d5fc39d6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:37.489650 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6f542-7442-4be9-9d92-051d5fc39d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:37.490012 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6f542-7442-4be9-9d92-051d5fc39d6a admin admin] PATH_INFO: `/v3/projects/44a9eff003944792a6a51c027004d8db` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:37.491154 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a6f542-7442-4be9-9d92-051d5fc39d6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=44a9eff003944792a6a51c027004d8db)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:37.493741 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a6f542-7442-4be9-9d92-051d5fc39d6a admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:37.546871 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:37.560559 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f4a6f542-7442-4be9-9d92-051d5fc39d6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44a9eff003944792a6a51c027004d8db', 'request_id': 'req-f4a6f542-7442-4be9-9d92-051d5fc39d6a'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:37.568817 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f4a6f542-7442-4be9-9d92-051d5fc39d6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44a9eff003944792a6a51c027004d8db', 'request_id': 'req-f4a6f542-7442-4be9-9d92-051d5fc39d6a'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:37.576388 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f4a6f542-7442-4be9-9d92-051d5fc39d6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44a9eff003944792a6a51c027004d8db', 'request_id': 'req-f4a6f542-7442-4be9-9d92-051d5fc39d6a'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:37.589077 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f4a6f542-7442-4be9-9d92-051d5fc39d6a admin admin] The token cache is being invalidate because project 44a9eff003944792a6a51c027004d8db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:41:37.592458 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f4a6f542-7442-4be9-9d92-051d5fc39d6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:37.593321 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f4a6f542-7442-4be9-9d92-051d5fc39d6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:37.595094 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1364/2744] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:41:37 2022] DELETE /identity/v3/projects/44a9eff003944792a6a51c027004d8db => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:37.599718 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028af4f9-4040-4d78-bb6a-12a6a4ee4a78 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:37.611617 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028af4f9-4040-4d78-bb6a-12a6a4ee4a78 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:37.678402 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e328f-2147-4ac7-9f28-8555b58136c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:37.679961 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e328f-2147-4ac7-9f28-8555b58136c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:37.680461 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e328f-2147-4ac7-9f28-8555b58136c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:37.680726 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e328f-2147-4ac7-9f28-8555b58136c8 admin admin] PATH_INFO: `/v3/users/7bb2de5b0bb2441ab0c52aa32400d265` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:37.682774 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894e328f-2147-4ac7-9f28-8555b58136c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bb2de5b0bb2441ab0c52aa32400d265)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:37.684783 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894e328f-2147-4ac7-9f28-8555b58136c8 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:37.743902 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-894e328f-2147-4ac7-9f28-8555b58136c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bb2de5b0bb2441ab0c52aa32400d265'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:37.757057 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-894e328f-2147-4ac7-9f28-8555b58136c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bb2de5b0bb2441ab0c52aa32400d265'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:37.766744 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-894e328f-2147-4ac7-9f28-8555b58136c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bb2de5b0bb2441ab0c52aa32400d265'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:37.767489 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-894e328f-2147-4ac7-9f28-8555b58136c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bb2de5b0bb2441ab0c52aa32400d265'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:37.768098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-894e328f-2147-4ac7-9f28-8555b58136c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bb2de5b0bb2441ab0c52aa32400d265'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:37.781659 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1381/2745] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:41:37 2022] DELETE /identity/v3/users/7bb2de5b0bb2441ab0c52aa32400d265 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:38.060742 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6f542-7442-4be9-9d92-051d5fc39d6a admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:38.062486 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-f4a6f542-7442-4be9-9d92-051d5fc39d6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:38.078260 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6f542-7442-4be9-9d92-051d5fc39d6a admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:38.142602 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61b9de-ae8c-4b23-b759-34ad53e7808e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:38.143470 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61b9de-ae8c-4b23-b759-34ad53e7808e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:38.143761 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61b9de-ae8c-4b23-b759-34ad53e7808e admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:38.144045 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61b9de-ae8c-4b23-b759-34ad53e7808e admin admin] PATH_INFO: `/v3/projects/56c7c4debeef4cb4be711bf1b9629006` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:38.145361 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61b9de-ae8c-4b23-b759-34ad53e7808e admin admin] RBAC: Authorizing `identity:delete_project(project_id=56c7c4debeef4cb4be711bf1b9629006)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:38.147303 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61b9de-ae8c-4b23-b759-34ad53e7808e admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:38.195029 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-ff61b9de-ae8c-4b23-b759-34ad53e7808e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56c7c4debeef4cb4be711bf1b9629006', 'request_id': 'req-ff61b9de-ae8c-4b23-b759-34ad53e7808e'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:38.202931 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-ff61b9de-ae8c-4b23-b759-34ad53e7808e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56c7c4debeef4cb4be711bf1b9629006', 'request_id': 'req-ff61b9de-ae8c-4b23-b759-34ad53e7808e'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:38.209982 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-ff61b9de-ae8c-4b23-b759-34ad53e7808e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56c7c4debeef4cb4be711bf1b9629006', 'request_id': 'req-ff61b9de-ae8c-4b23-b759-34ad53e7808e'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:38.220668 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-ff61b9de-ae8c-4b23-b759-34ad53e7808e admin admin] The token cache is being invalidate because project 56c7c4debeef4cb4be711bf1b9629006 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:41:38.220960 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-ff61b9de-ae8c-4b23-b759-34ad53e7808e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:38.221613 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-ff61b9de-ae8c-4b23-b759-34ad53e7808e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:38.223133 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1365/2746] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:41:38 2022] DELETE /identity/v3/projects/56c7c4debeef4cb4be711bf1b9629006 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:38.228833 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d3e84f-d507-49a6-84ca-65f40d2efd47 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:38.229196 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d3e84f-d507-49a6-84ca-65f40d2efd47 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:38.229522 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d3e84f-d507-49a6-84ca-65f40d2efd47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:38.250455 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-91d3e84f-d507-49a6-84ca-65f40d2efd47 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:38.251654 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-91d3e84f-d507-49a6-84ca-65f40d2efd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:38.296238 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:38.317158 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1382/2747] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:41:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:41:38.326182 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101effc9-fd9c-415c-8521-b1509ce8e58c None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:38.326482 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101effc9-fd9c-415c-8521-b1509ce8e58c None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:38.326767 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101effc9-fd9c-415c-8521-b1509ce8e58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:38.347918 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-101effc9-fd9c-415c-8521-b1509ce8e58c None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:38.349284 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-101effc9-fd9c-415c-8521-b1509ce8e58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:38.358182 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1366/2748] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:41:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:41:38.362534 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d3e84f-d507-49a6-84ca-65f40d2efd47 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:38.372826 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d3e84f-d507-49a6-84ca-65f40d2efd47 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:38.377324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0a5cf-5690-4840-bb19-a7c73c591ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:38.377997 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0a5cf-5690-4840-bb19-a7c73c591ebb admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:38.378264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0a5cf-5690-4840-bb19-a7c73c591ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:38.378541 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0a5cf-5690-4840-bb19-a7c73c591ebb admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:38.379266 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0a5cf-5690-4840-bb19-a7c73c591ebb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:41:38.379574 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0a5cf-5690-4840-bb19-a7c73c591ebb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:38.381188 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0a5cf-5690-4840-bb19-a7c73c591ebb admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:38.396380 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1383/2749] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:41:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:41:38.397510 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101effc9-fd9c-415c-8521-b1509ce8e58c None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:38.407368 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101effc9-fd9c-415c-8521-b1509ce8e58c None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:38.412612 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b85098-92b2-48c4-9356-a8cb27dadf8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:38.413264 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b85098-92b2-48c4-9356-a8cb27dadf8a admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:38.414183 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b85098-92b2-48c4-9356-a8cb27dadf8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:38.414484 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b85098-92b2-48c4-9356-a8cb27dadf8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:38.415198 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b85098-92b2-48c4-9356-a8cb27dadf8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:38.417306 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b85098-92b2-48c4-9356-a8cb27dadf8a admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:38.450226 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1367/2750] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:41:38 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:41:38.463049 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0a5cf-5690-4840-bb19-a7c73c591ebb admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:38.465493 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0a5cf-5690-4840-bb19-a7c73c591ebb admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:38.524636 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77308c9e-f3e4-48a8-9934-0c2730ab693b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:38.525274 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77308c9e-f3e4-48a8-9934-0c2730ab693b admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:38.525557 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77308c9e-f3e4-48a8-9934-0c2730ab693b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:38.525836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77308c9e-f3e4-48a8-9934-0c2730ab693b admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:38.526517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77308c9e-f3e4-48a8-9934-0c2730ab693b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:38.528461 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77308c9e-f3e4-48a8-9934-0c2730ab693b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:38.550413 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1384/2751] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:41:38 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:41:38.555289 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b85098-92b2-48c4-9356-a8cb27dadf8a admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:38.567052 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b85098-92b2-48c4-9356-a8cb27dadf8a admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:38.572022 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2d005-7d25-46e8-aea8-4d2a5df3c59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:38.572713 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2d005-7d25-46e8-aea8-4d2a5df3c59a admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:38.573113 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2d005-7d25-46e8-aea8-4d2a5df3c59a admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:38.573396 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2d005-7d25-46e8-aea8-4d2a5df3c59a admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:38.574116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a2d005-7d25-46e8-aea8-4d2a5df3c59a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:38.575710 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a2d005-7d25-46e8-aea8-4d2a5df3c59a admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:38.587305 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1368/2752] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:41:38.591600 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77308c9e-f3e4-48a8-9934-0c2730ab693b admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:38.602513 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77308c9e-f3e4-48a8-9934-0c2730ab693b admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:38.607260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef258f-4c7a-443d-bd7d-9535fbb3b927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:38.607921 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef258f-4c7a-443d-bd7d-9535fbb3b927 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:38.608303 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef258f-4c7a-443d-bd7d-9535fbb3b927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:38.608599 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef258f-4c7a-443d-bd7d-9535fbb3b927 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:38.609403 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef258f-4c7a-443d-bd7d-9535fbb3b927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:38.610927 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef258f-4c7a-443d-bd7d-9535fbb3b927 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:38.622980 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1385/2753] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:41:38.627151 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2d005-7d25-46e8-aea8-4d2a5df3c59a admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:38.637918 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2d005-7d25-46e8-aea8-4d2a5df3c59a admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:38.643045 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37acc01d-d70b-417f-b8aa-dc5cb121388b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:38.644007 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acc01d-d70b-417f-b8aa-dc5cb121388b admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:38.644325 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acc01d-d70b-417f-b8aa-dc5cb121388b admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:38.644606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acc01d-d70b-417f-b8aa-dc5cb121388b admin admin] PATH_INFO: `/v3/projects/962ee58f63fc461a8cceac249fd91d2e/users/8e053a56af614eff85b1a907d5b21f0c/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:38.659594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:38.663362 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37acc01d-d70b-417f-b8aa-dc5cb121388b admin admin] RBAC: Authorizing `identity:create_grant(project_id=962ee58f63fc461a8cceac249fd91d2e, user_id=8e053a56af614eff85b1a907d5b21f0c, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:38.668164 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37acc01d-d70b-417f-b8aa-dc5cb121388b admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:38.683590 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1369/2754] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:41:38 2022] PUT /identity/v3/projects/962ee58f63fc461a8cceac249fd91d2e/users/8e053a56af614eff85b1a907d5b21f0c/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:38.694082 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f0f46-63fa-4715-9119-3d8325f066bc None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:38.694377 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f0f46-63fa-4715-9119-3d8325f066bc None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:38.694668 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f0f46-63fa-4715-9119-3d8325f066bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:38.722783 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-7b6f0f46-63fa-4715-9119-3d8325f066bc None None] MFA Rules not processed for user `8e053a56af614eff85b1a907d5b21f0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:38.724014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-7b6f0f46-63fa-4715-9119-3d8325f066bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:38.791754 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1386/2755] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:41:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:41:38.796413 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37acc01d-d70b-417f-b8aa-dc5cb121388b admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:38.863951 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37acc01d-d70b-417f-b8aa-dc5cb121388b admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:38.869901 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404ba5f-ed8a-405a-ac6b-c345f808ab5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:38.870575 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404ba5f-ed8a-405a-ac6b-c345f808ab5e admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:38.870864 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404ba5f-ed8a-405a-ac6b-c345f808ab5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:38.871147 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404ba5f-ed8a-405a-ac6b-c345f808ab5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:38.871855 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404ba5f-ed8a-405a-ac6b-c345f808ab5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:38.873960 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a404ba5f-ed8a-405a-ac6b-c345f808ab5e admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:38.900127 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1370/2756] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:41:38 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:41:38.904333 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f0f46-63fa-4715-9119-3d8325f066bc None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:38.917189 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f0f46-63fa-4715-9119-3d8325f066bc None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:38.958248 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:38.977241 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc5fc2-3b80-498f-9b36-275bee761668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:38.977937 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc5fc2-3b80-498f-9b36-275bee761668 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:38.978227 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc5fc2-3b80-498f-9b36-275bee761668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:38.978510 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc5fc2-3b80-498f-9b36-275bee761668 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:38.979207 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc5fc2-3b80-498f-9b36-275bee761668 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:38.981205 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc5fc2-3b80-498f-9b36-275bee761668 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:39.002586 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1387/2757] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:41:38 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:41:39.006687 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404ba5f-ed8a-405a-ac6b-c345f808ab5e admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:39.017135 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404ba5f-ed8a-405a-ac6b-c345f808ab5e admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:39.022211 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fbf2e7-ead7-452a-8096-a9b1aec326eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:39.022998 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fbf2e7-ead7-452a-8096-a9b1aec326eb admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:39.023308 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fbf2e7-ead7-452a-8096-a9b1aec326eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:39.023688 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fbf2e7-ead7-452a-8096-a9b1aec326eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:39.024669 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fbf2e7-ead7-452a-8096-a9b1aec326eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:39.026337 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fbf2e7-ead7-452a-8096-a9b1aec326eb admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:39.037740 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1371/2758] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:41:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:41:39.042555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc5fc2-3b80-498f-9b36-275bee761668 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:39.052541 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc5fc2-3b80-498f-9b36-275bee761668 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:39.060765 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64de601-1f22-4592-99d2-3ddafd26f551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:39.061469 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64de601-1f22-4592-99d2-3ddafd26f551 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:39.061989 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64de601-1f22-4592-99d2-3ddafd26f551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:39.062164 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64de601-1f22-4592-99d2-3ddafd26f551 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:39.062764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64de601-1f22-4592-99d2-3ddafd26f551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:39.064592 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64de601-1f22-4592-99d2-3ddafd26f551 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:39.075736 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1388/2759] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:41:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:41:39.080462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fbf2e7-ead7-452a-8096-a9b1aec326eb admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:39.091957 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fbf2e7-ead7-452a-8096-a9b1aec326eb admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:39.096633 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426200e-ec84-4184-9005-8c0ffb7eb6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:39.097533 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426200e-ec84-4184-9005-8c0ffb7eb6b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:39.097820 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426200e-ec84-4184-9005-8c0ffb7eb6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:39.098099 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426200e-ec84-4184-9005-8c0ffb7eb6b0 admin admin] PATH_INFO: `/v3/projects/fe3188038094486d9f72b7e791ad4330/users/6e3543188c564fad8cb56f7fbeb25b44/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:39.113768 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c426200e-ec84-4184-9005-8c0ffb7eb6b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe3188038094486d9f72b7e791ad4330, user_id=6e3543188c564fad8cb56f7fbeb25b44, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:39.116540 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c426200e-ec84-4184-9005-8c0ffb7eb6b0 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:39.130094 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1372/2760] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:41:39 2022] PUT /identity/v3/projects/fe3188038094486d9f72b7e791ad4330/users/6e3543188c564fad8cb56f7fbeb25b44/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:39.140892 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739270b-6bda-4b10-a664-98514c880da4 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:39.141171 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739270b-6bda-4b10-a664-98514c880da4 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:39.141454 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739270b-6bda-4b10-a664-98514c880da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:39.171412 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-5739270b-6bda-4b10-a664-98514c880da4 None None] MFA Rules not processed for user `6e3543188c564fad8cb56f7fbeb25b44`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:39.172669 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5739270b-6bda-4b10-a664-98514c880da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:39.237928 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1389/2761] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:41:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:41:39.242871 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426200e-ec84-4184-9005-8c0ffb7eb6b0 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:39.307366 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426200e-ec84-4184-9005-8c0ffb7eb6b0 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:39.312093 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc1833-3b98-448c-8bac-ce7f4e0ec698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:39.312786 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc1833-3b98-448c-8bac-ce7f4e0ec698 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:39.313073 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc1833-3b98-448c-8bac-ce7f4e0ec698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:39.313381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc1833-3b98-448c-8bac-ce7f4e0ec698 admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:39.314085 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fc1833-3b98-448c-8bac-ce7f4e0ec698 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:39.316037 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fc1833-3b98-448c-8bac-ce7f4e0ec698 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:39.343583 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1373/2762] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:41:39 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:41:39.348318 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739270b-6bda-4b10-a664-98514c880da4 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:39.360329 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739270b-6bda-4b10-a664-98514c880da4 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:39.419254 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218573ff-bf75-4cb4-be3a-48662aebf3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:39.419843 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218573ff-bf75-4cb4-be3a-48662aebf3c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:39.420129 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218573ff-bf75-4cb4-be3a-48662aebf3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:39.420437 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218573ff-bf75-4cb4-be3a-48662aebf3c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:39.421142 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218573ff-bf75-4cb4-be3a-48662aebf3c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:39.423428 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218573ff-bf75-4cb4-be3a-48662aebf3c3 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:39.445338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1390/2763] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:41:39 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:41:39.448733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc1833-3b98-448c-8bac-ce7f4e0ec698 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:39.460122 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc1833-3b98-448c-8bac-ce7f4e0ec698 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:39.465065 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9e333a-62f9-4a96-bc10-d6443f273ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:39.468299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e333a-62f9-4a96-bc10-d6443f273ca8 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:39.468609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e333a-62f9-4a96-bc10-d6443f273ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:39.468904 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9e333a-62f9-4a96-bc10-d6443f273ca8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:39.469641 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9e333a-62f9-4a96-bc10-d6443f273ca8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:39.471244 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9e333a-62f9-4a96-bc10-d6443f273ca8 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:39.483651 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1374/2764] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:41:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:41:39.487672 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218573ff-bf75-4cb4-be3a-48662aebf3c3 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:39.498201 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218573ff-bf75-4cb4-be3a-48662aebf3c3 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:39.505097 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0c097-aa25-44ad-b3b0-d67b6c900ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:39.505995 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0c097-aa25-44ad-b3b0-d67b6c900ed4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:39.506403 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0c097-aa25-44ad-b3b0-d67b6c900ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:39.507159 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0c097-aa25-44ad-b3b0-d67b6c900ed4 admin admin] PATH_INFO: `/v3/projects/f441f1bd197c4b8b93bd7d3a001208b3/users/142f0823155b4e2e90e5d63516b72a96/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:39.523605 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0c097-aa25-44ad-b3b0-d67b6c900ed4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f441f1bd197c4b8b93bd7d3a001208b3, user_id=142f0823155b4e2e90e5d63516b72a96, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:39.526280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0c097-aa25-44ad-b3b0-d67b6c900ed4 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:39.540077 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1391/2765] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:41:39 2022] PUT /identity/v3/projects/f441f1bd197c4b8b93bd7d3a001208b3/users/142f0823155b4e2e90e5d63516b72a96/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:39.544371 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9e333a-62f9-4a96-bc10-d6443f273ca8 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:39.555482 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9e333a-62f9-4a96-bc10-d6443f273ca8 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:39.567304 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:39.619808 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189bcb3-4306-40e5-8a79-048d1b703d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:39.620528 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189bcb3-4306-40e5-8a79-048d1b703d54 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:39.620817 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189bcb3-4306-40e5-8a79-048d1b703d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:39.621098 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189bcb3-4306-40e5-8a79-048d1b703d54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:39.621839 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189bcb3-4306-40e5-8a79-048d1b703d54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:39.623459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189bcb3-4306-40e5-8a79-048d1b703d54 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:39.634818 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1375/2766] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:41:39 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:41:39.639160 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0c097-aa25-44ad-b3b0-d67b6c900ed4 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:39.650570 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0c097-aa25-44ad-b3b0-d67b6c900ed4 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:39.655494 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e660c43-0741-445c-b581-ec94f15fda39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:39.656417 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e660c43-0741-445c-b581-ec94f15fda39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:39.656701 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e660c43-0741-445c-b581-ec94f15fda39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:39.656981 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e660c43-0741-445c-b581-ec94f15fda39 admin admin] PATH_INFO: `/v3/projects/f441f1bd197c4b8b93bd7d3a001208b3/users/142f0823155b4e2e90e5d63516b72a96/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:39.659042 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e660c43-0741-445c-b581-ec94f15fda39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f441f1bd197c4b8b93bd7d3a001208b3, user_id=142f0823155b4e2e90e5d63516b72a96, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:39.661731 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e660c43-0741-445c-b581-ec94f15fda39 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:39.674018 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1392/2767] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:41:39 2022] PUT /identity/v3/projects/f441f1bd197c4b8b93bd7d3a001208b3/users/142f0823155b4e2e90e5d63516b72a96/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:39.689738 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b17633-09d9-476c-aac0-e56a0f44bbe2 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:39.690036 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b17633-09d9-476c-aac0-e56a0f44bbe2 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:39.690343 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b17633-09d9-476c-aac0-e56a0f44bbe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:39.719175 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-60b17633-09d9-476c-aac0-e56a0f44bbe2 None None] MFA Rules not processed for user `142f0823155b4e2e90e5d63516b72a96`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:39.720423 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-60b17633-09d9-476c-aac0-e56a0f44bbe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:39.792239 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1376/2768] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:41:39 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:41:39.803715 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e660c43-0741-445c-b581-ec94f15fda39 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:39.806564 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3e660c43-0741-445c-b581-ec94f15fda39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:39.811935 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:39.868394 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e660c43-0741-445c-b581-ec94f15fda39 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:39.872023 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f63df2-3616-4ce5-ab3f-45d00ff87026 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:39.872705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f63df2-3616-4ce5-ab3f-45d00ff87026 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:39.872985 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f63df2-3616-4ce5-ab3f-45d00ff87026 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:39.873260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f63df2-3616-4ce5-ab3f-45d00ff87026 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:39.940319 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f63df2-3616-4ce5-ab3f-45d00ff87026 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:39.941832 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f63df2-3616-4ce5-ab3f-45d00ff87026 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:39.952966 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1393/2769] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:41:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:41:41.472776 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b17633-09d9-476c-aac0-e56a0f44bbe2 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:41.482977 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b17633-09d9-476c-aac0-e56a0f44bbe2 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:41.486646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274ab06-6596-4bf3-ae6a-02cd8fc2a86b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:41.487288 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274ab06-6596-4bf3-ae6a-02cd8fc2a86b service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:41.487572 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274ab06-6596-4bf3-ae6a-02cd8fc2a86b service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:41.487847 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274ab06-6596-4bf3-ae6a-02cd8fc2a86b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:41.547346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274ab06-6596-4bf3-ae6a-02cd8fc2a86b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:41.548759 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274ab06-6596-4bf3-ae6a-02cd8fc2a86b service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:41.559010 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1377/2770] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:41:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:41:42.138198 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f63df2-3616-4ce5-ab3f-45d00ff87026 service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:42.150022 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f63df2-3616-4ce5-ab3f-45d00ff87026 service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:42.153764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e760c0e-ceba-468d-9bda-a48614974b07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:42.154421 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e760c0e-ceba-468d-9bda-a48614974b07 service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:42.154707 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e760c0e-ceba-468d-9bda-a48614974b07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:42.155029 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e760c0e-ceba-468d-9bda-a48614974b07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:42.169202 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e760c0e-ceba-468d-9bda-a48614974b07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:42.170704 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e760c0e-ceba-468d-9bda-a48614974b07 service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:42.182074 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1394/2771] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:41:42 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:41:46.090408 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274ab06-6596-4bf3-ae6a-02cd8fc2a86b service cinder] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:46.130056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:46.157037 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274ab06-6596-4bf3-ae6a-02cd8fc2a86b service cinder] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:46.161585 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582b24c-0a91-4fc4-9c36-9665e9648c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:46.162460 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582b24c-0a91-4fc4-9c36-9665e9648c83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:46.162745 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582b24c-0a91-4fc4-9c36-9665e9648c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:46.163027 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582b24c-0a91-4fc4-9c36-9665e9648c83 admin admin] PATH_INFO: `/v3/users/8e053a56af614eff85b1a907d5b21f0c` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:46.164239 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582b24c-0a91-4fc4-9c36-9665e9648c83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e053a56af614eff85b1a907d5b21f0c)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:46.166421 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582b24c-0a91-4fc4-9c36-9665e9648c83 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:46.215664 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f582b24c-0a91-4fc4-9c36-9665e9648c83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e053a56af614eff85b1a907d5b21f0c'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:46.216350 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f582b24c-0a91-4fc4-9c36-9665e9648c83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e053a56af614eff85b1a907d5b21f0c'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:46.223261 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f582b24c-0a91-4fc4-9c36-9665e9648c83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e053a56af614eff85b1a907d5b21f0c'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:46.235493 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f582b24c-0a91-4fc4-9c36-9665e9648c83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e053a56af614eff85b1a907d5b21f0c'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:46.243631 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-f582b24c-0a91-4fc4-9c36-9665e9648c83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e053a56af614eff85b1a907d5b21f0c'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:46.250991 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1378/2772] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:41:46 2022] DELETE /identity/v3/users/8e053a56af614eff85b1a907d5b21f0c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:46.255986 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc34c29-0505-4afb-80bb-88aadb1e27f9 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:46.256597 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc34c29-0505-4afb-80bb-88aadb1e27f9 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:46.256597 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc34c29-0505-4afb-80bb-88aadb1e27f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:46.277114 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-7dc34c29-0505-4afb-80bb-88aadb1e27f9 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:46.278235 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-7dc34c29-0505-4afb-80bb-88aadb1e27f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:46.340367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1395/2773] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:41:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:41:46.350186 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582b24c-0a91-4fc4-9c36-9665e9648c83 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:46.351602 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-f582b24c-0a91-4fc4-9c36-9665e9648c83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:46.410040 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582b24c-0a91-4fc4-9c36-9665e9648c83 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:46.413701 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13923fb-36d0-4a26-8c21-3925ec3f30d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:46.414349 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13923fb-36d0-4a26-8c21-3925ec3f30d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:46.414646 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13923fb-36d0-4a26-8c21-3925ec3f30d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:46.414960 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13923fb-36d0-4a26-8c21-3925ec3f30d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:46.424400 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13923fb-36d0-4a26-8c21-3925ec3f30d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:46.425888 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13923fb-36d0-4a26-8c21-3925ec3f30d3 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:46.436465 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1379/2774] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:41:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:41:46.721119 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc34c29-0505-4afb-80bb-88aadb1e27f9 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:46.722717 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-7dc34c29-0505-4afb-80bb-88aadb1e27f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:46.735943 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc34c29-0505-4afb-80bb-88aadb1e27f9 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:46.740414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3379f46-3f2b-41e1-8223-c06f86c7445e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:46.741302 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3379f46-3f2b-41e1-8223-c06f86c7445e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:46.741583 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3379f46-3f2b-41e1-8223-c06f86c7445e admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:46.741862 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3379f46-3f2b-41e1-8223-c06f86c7445e admin admin] PATH_INFO: `/v3/projects/962ee58f63fc461a8cceac249fd91d2e` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:46.744287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3379f46-3f2b-41e1-8223-c06f86c7445e admin admin] RBAC: Authorizing `identity:delete_project(project_id=962ee58f63fc461a8cceac249fd91d2e)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:46.746185 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3379f46-3f2b-41e1-8223-c06f86c7445e admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:46.785982 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:46.794656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a3379f46-3f2b-41e1-8223-c06f86c7445e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '962ee58f63fc461a8cceac249fd91d2e', 'request_id': 'req-a3379f46-3f2b-41e1-8223-c06f86c7445e'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:46.801015 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a3379f46-3f2b-41e1-8223-c06f86c7445e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '962ee58f63fc461a8cceac249fd91d2e', 'request_id': 'req-a3379f46-3f2b-41e1-8223-c06f86c7445e'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:46.807930 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a3379f46-3f2b-41e1-8223-c06f86c7445e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '962ee58f63fc461a8cceac249fd91d2e', 'request_id': 'req-a3379f46-3f2b-41e1-8223-c06f86c7445e'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:46.819230 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a3379f46-3f2b-41e1-8223-c06f86c7445e admin admin] The token cache is being invalidate because project 962ee58f63fc461a8cceac249fd91d2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:41:46.819516 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a3379f46-3f2b-41e1-8223-c06f86c7445e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:46.820125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a3379f46-3f2b-41e1-8223-c06f86c7445e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:46.821709 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1396/2775] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:41:46 2022] DELETE /identity/v3/projects/962ee58f63fc461a8cceac249fd91d2e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:46.825639 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13923fb-36d0-4a26-8c21-3925ec3f30d3 service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:46.835659 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13923fb-36d0-4a26-8c21-3925ec3f30d3 service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:46.893923 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:46.901303 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71889e5d-4f66-41b9-9cb4-efa9620ba2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:46.904604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71889e5d-4f66-41b9-9cb4-efa9620ba2eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:46.905018 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71889e5d-4f66-41b9-9cb4-efa9620ba2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:46.905540 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71889e5d-4f66-41b9-9cb4-efa9620ba2eb admin admin] PATH_INFO: `/v3/users/6e3543188c564fad8cb56f7fbeb25b44` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:46.907614 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71889e5d-4f66-41b9-9cb4-efa9620ba2eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e3543188c564fad8cb56f7fbeb25b44)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:46.909702 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71889e5d-4f66-41b9-9cb4-efa9620ba2eb admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:46.957971 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-71889e5d-4f66-41b9-9cb4-efa9620ba2eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e3543188c564fad8cb56f7fbeb25b44'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:46.958627 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-71889e5d-4f66-41b9-9cb4-efa9620ba2eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e3543188c564fad8cb56f7fbeb25b44'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:46.966288 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-71889e5d-4f66-41b9-9cb4-efa9620ba2eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e3543188c564fad8cb56f7fbeb25b44'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:46.978896 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-71889e5d-4f66-41b9-9cb4-efa9620ba2eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e3543188c564fad8cb56f7fbeb25b44'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:46.985921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-71889e5d-4f66-41b9-9cb4-efa9620ba2eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e3543188c564fad8cb56f7fbeb25b44'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:46.992280 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1380/2776] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:41:46 2022] DELETE /identity/v3/users/6e3543188c564fad8cb56f7fbeb25b44 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:47.269917 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3379f46-3f2b-41e1-8223-c06f86c7445e admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:47.271450 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a3379f46-3f2b-41e1-8223-c06f86c7445e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:47.285932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3379f46-3f2b-41e1-8223-c06f86c7445e admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:47.350821 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0946d2e-4798-4292-9372-9e036af81180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:47.351600 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0946d2e-4798-4292-9372-9e036af81180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:47.351885 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0946d2e-4798-4292-9372-9e036af81180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:47.352272 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0946d2e-4798-4292-9372-9e036af81180 admin admin] PATH_INFO: `/v3/projects/fe3188038094486d9f72b7e791ad4330` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:47.353704 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0946d2e-4798-4292-9372-9e036af81180 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe3188038094486d9f72b7e791ad4330)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:47.355612 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0946d2e-4798-4292-9372-9e036af81180 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:47.402485 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e0946d2e-4798-4292-9372-9e036af81180 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe3188038094486d9f72b7e791ad4330', 'request_id': 'req-e0946d2e-4798-4292-9372-9e036af81180'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:47.408025 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e0946d2e-4798-4292-9372-9e036af81180 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe3188038094486d9f72b7e791ad4330', 'request_id': 'req-e0946d2e-4798-4292-9372-9e036af81180'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:47.414143 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e0946d2e-4798-4292-9372-9e036af81180 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe3188038094486d9f72b7e791ad4330', 'request_id': 'req-e0946d2e-4798-4292-9372-9e036af81180'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:47.425700 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e0946d2e-4798-4292-9372-9e036af81180 admin admin] The token cache is being invalidate because project fe3188038094486d9f72b7e791ad4330 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:41:47.425991 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e0946d2e-4798-4292-9372-9e036af81180 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:47.426628 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e0946d2e-4798-4292-9372-9e036af81180 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:47.428153 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1397/2777] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:41:47 2022] DELETE /identity/v3/projects/fe3188038094486d9f72b7e791ad4330 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:47.432526 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71889e5d-4f66-41b9-9cb4-efa9620ba2eb admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:47.433991 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-71889e5d-4f66-41b9-9cb4-efa9620ba2eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:47.447136 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71889e5d-4f66-41b9-9cb4-efa9620ba2eb admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:47.502542 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec2549-c5f8-4591-95ff-d1143f395150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:47.503413 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec2549-c5f8-4591-95ff-d1143f395150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:47.503704 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec2549-c5f8-4591-95ff-d1143f395150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:47.503988 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec2549-c5f8-4591-95ff-d1143f395150 admin admin] PATH_INFO: `/v3/users/142f0823155b4e2e90e5d63516b72a96` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:47.505214 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec2549-c5f8-4591-95ff-d1143f395150 admin admin] RBAC: Authorizing `identity:delete_user(user_id=142f0823155b4e2e90e5d63516b72a96)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:47.507130 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec2549-c5f8-4591-95ff-d1143f395150 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:47.553167 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-3fec2549-c5f8-4591-95ff-d1143f395150 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '142f0823155b4e2e90e5d63516b72a96'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:47.553828 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-3fec2549-c5f8-4591-95ff-d1143f395150 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '142f0823155b4e2e90e5d63516b72a96'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:47.560862 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-3fec2549-c5f8-4591-95ff-d1143f395150 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '142f0823155b4e2e90e5d63516b72a96'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:47.573506 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-3fec2549-c5f8-4591-95ff-d1143f395150 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '142f0823155b4e2e90e5d63516b72a96'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:47.577103 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:47.581558 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-3fec2549-c5f8-4591-95ff-d1143f395150 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '142f0823155b4e2e90e5d63516b72a96'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:47.589783 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1381/2778] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:41:47 2022] DELETE /identity/v3/users/142f0823155b4e2e90e5d63516b72a96 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:47.859820 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0946d2e-4798-4292-9372-9e036af81180 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:47.861492 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e0946d2e-4798-4292-9372-9e036af81180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:47.874327 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0946d2e-4798-4292-9372-9e036af81180 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:47.934222 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:47.941517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2970d9-6161-4e04-87b3-8518e181abbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:47.942590 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2970d9-6161-4e04-87b3-8518e181abbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:47.943094 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2970d9-6161-4e04-87b3-8518e181abbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:47.943576 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2970d9-6161-4e04-87b3-8518e181abbe admin admin] PATH_INFO: `/v3/projects/f441f1bd197c4b8b93bd7d3a001208b3` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:47.944929 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2970d9-6161-4e04-87b3-8518e181abbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=f441f1bd197c4b8b93bd7d3a001208b3)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:47.947820 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2970d9-6161-4e04-87b3-8518e181abbe admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:47.994783 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5d2970d9-6161-4e04-87b3-8518e181abbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f441f1bd197c4b8b93bd7d3a001208b3', 'request_id': 'req-5d2970d9-6161-4e04-87b3-8518e181abbe'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:48.000051 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5d2970d9-6161-4e04-87b3-8518e181abbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f441f1bd197c4b8b93bd7d3a001208b3', 'request_id': 'req-5d2970d9-6161-4e04-87b3-8518e181abbe'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:48.006081 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5d2970d9-6161-4e04-87b3-8518e181abbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f441f1bd197c4b8b93bd7d3a001208b3', 'request_id': 'req-5d2970d9-6161-4e04-87b3-8518e181abbe'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:48.018809 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5d2970d9-6161-4e04-87b3-8518e181abbe admin admin] The token cache is being invalidate because project f441f1bd197c4b8b93bd7d3a001208b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80215) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:41:48.019094 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5d2970d9-6161-4e04-87b3-8518e181abbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:48.019709 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-5d2970d9-6161-4e04-87b3-8518e181abbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:48.021266 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1398/2779] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:41:47 2022] DELETE /identity/v3/projects/f441f1bd197c4b8b93bd7d3a001208b3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:48.027024 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e23de5-8a3e-45da-bf33-127d67b2113a None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:48.027329 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e23de5-8a3e-45da-bf33-127d67b2113a None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:48.027624 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e23de5-8a3e-45da-bf33-127d67b2113a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:48.046991 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-60e23de5-8a3e-45da-bf33-127d67b2113a None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:48.048344 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-60e23de5-8a3e-45da-bf33-127d67b2113a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:48.107256 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1382/2780] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:41:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:41:48.115109 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426241bf-09d2-4867-ac15-842240a2bea9 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:48.115328 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426241bf-09d2-4867-ac15-842240a2bea9 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:48.115888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426241bf-09d2-4867-ac15-842240a2bea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:48.135481 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-426241bf-09d2-4867-ac15-842240a2bea9 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:48.136724 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-426241bf-09d2-4867-ac15-842240a2bea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:48.144619 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1399/2781] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:41:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:41:48.148691 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e23de5-8a3e-45da-bf33-127d67b2113a None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:48.158302 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e23de5-8a3e-45da-bf33-127d67b2113a None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:48.162931 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c47b9-03ec-4927-81f6-422ca1b147f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:48.163582 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c47b9-03ec-4927-81f6-422ca1b147f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:48.163874 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c47b9-03ec-4927-81f6-422ca1b147f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:48.164158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c47b9-03ec-4927-81f6-422ca1b147f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:48.165092 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c47b9-03ec-4927-81f6-422ca1b147f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80216) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:41:48.165447 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c47b9-03ec-4927-81f6-422ca1b147f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:48.167060 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6c47b9-03ec-4927-81f6-422ca1b147f4 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:48.178635 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1383/2782] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:41:48.183775 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426241bf-09d2-4867-ac15-842240a2bea9 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:48.193061 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426241bf-09d2-4867-ac15-842240a2bea9 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:48.198559 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd316bd-fddf-467e-a521-58be21f094e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:48.199208 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd316bd-fddf-467e-a521-58be21f094e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:48.199492 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd316bd-fddf-467e-a521-58be21f094e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:48.199769 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd316bd-fddf-467e-a521-58be21f094e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:48.200488 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd316bd-fddf-467e-a521-58be21f094e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:48.202385 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd316bd-fddf-467e-a521-58be21f094e3 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:48.227792 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1400/2783] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:41:48 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:41:48.231770 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c47b9-03ec-4927-81f6-422ca1b147f4 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:48.241731 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c47b9-03ec-4927-81f6-422ca1b147f4 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:48.300932 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43310387-4619-44be-af1d-2d74bcf93c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:48.301606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43310387-4619-44be-af1d-2d74bcf93c1c admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:48.301898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43310387-4619-44be-af1d-2d74bcf93c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:48.302189 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43310387-4619-44be-af1d-2d74bcf93c1c admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:48.302874 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43310387-4619-44be-af1d-2d74bcf93c1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:48.304889 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43310387-4619-44be-af1d-2d74bcf93c1c admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:48.325839 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1384/2784] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:41:48 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:41:48.330196 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd316bd-fddf-467e-a521-58be21f094e3 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:48.340764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd316bd-fddf-467e-a521-58be21f094e3 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:48.345383 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a2cf3-3356-43aa-8d63-6d2a9ede9753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:48.346043 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a2cf3-3356-43aa-8d63-6d2a9ede9753 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:48.346338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a2cf3-3356-43aa-8d63-6d2a9ede9753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:48.346619 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a2cf3-3356-43aa-8d63-6d2a9ede9753 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:48.347343 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a2cf3-3356-43aa-8d63-6d2a9ede9753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:48.348974 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a2cf3-3356-43aa-8d63-6d2a9ede9753 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:48.359299 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1401/2785] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:41:48.363187 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43310387-4619-44be-af1d-2d74bcf93c1c admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:48.374245 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43310387-4619-44be-af1d-2d74bcf93c1c admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:48.378905 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79770626-2a9f-4ccb-9154-f9c62e1f55b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:48.379549 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79770626-2a9f-4ccb-9154-f9c62e1f55b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:48.379839 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79770626-2a9f-4ccb-9154-f9c62e1f55b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:48.380125 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79770626-2a9f-4ccb-9154-f9c62e1f55b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:48.380889 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79770626-2a9f-4ccb-9154-f9c62e1f55b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:48.382490 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79770626-2a9f-4ccb-9154-f9c62e1f55b1 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:48.393116 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1385/2786] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:41:48.397422 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a2cf3-3356-43aa-8d63-6d2a9ede9753 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:48.407515 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a2cf3-3356-43aa-8d63-6d2a9ede9753 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:48.412285 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c07b0-f77c-4767-9176-a7ffe74509f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:48.413167 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c07b0-f77c-4767-9176-a7ffe74509f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:48.413459 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c07b0-f77c-4767-9176-a7ffe74509f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:48.413803 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c07b0-f77c-4767-9176-a7ffe74509f2 admin admin] PATH_INFO: `/v3/projects/0cc7b8eb4a4a4312a4d104681ab640b4/users/98c568af160746ea948dfe8c54b6fe6f/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:48.429041 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70c07b0-f77c-4767-9176-a7ffe74509f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cc7b8eb4a4a4312a4d104681ab640b4, user_id=98c568af160746ea948dfe8c54b6fe6f, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:48.432237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70c07b0-f77c-4767-9176-a7ffe74509f2 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:48.444764 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1402/2787] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:41:48 2022] PUT /identity/v3/projects/0cc7b8eb4a4a4312a4d104681ab640b4/users/98c568af160746ea948dfe8c54b6fe6f/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:48.456689 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81983f2c-d877-4379-ad2a-0a8822a6af82 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:48.457182 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81983f2c-d877-4379-ad2a-0a8822a6af82 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:48.457278 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81983f2c-d877-4379-ad2a-0a8822a6af82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:48.484383 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-81983f2c-d877-4379-ad2a-0a8822a6af82 None None] MFA Rules not processed for user `98c568af160746ea948dfe8c54b6fe6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:48.485582 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-81983f2c-d877-4379-ad2a-0a8822a6af82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:48.532164 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:48.550262 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1386/2788] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:41:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:41:48.554862 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c07b0-f77c-4767-9176-a7ffe74509f2 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:48.616537 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c07b0-f77c-4767-9176-a7ffe74509f2 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:48.621927 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc86b3-dd23-4565-8df0-3ae9491ef536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:48.622634 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc86b3-dd23-4565-8df0-3ae9491ef536 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:48.622923 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc86b3-dd23-4565-8df0-3ae9491ef536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:48.623200 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc86b3-dd23-4565-8df0-3ae9491ef536 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:48.623881 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc86b3-dd23-4565-8df0-3ae9491ef536 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:48.625940 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc86b3-dd23-4565-8df0-3ae9491ef536 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:48.650414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1403/2789] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:41:48 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:41:48.654444 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81983f2c-d877-4379-ad2a-0a8822a6af82 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:48.666282 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81983f2c-d877-4379-ad2a-0a8822a6af82 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:48.722054 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2342e-f6fd-4ad7-8682-154fd0bfc32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:48.722742 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2342e-f6fd-4ad7-8682-154fd0bfc32c admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:48.723024 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2342e-f6fd-4ad7-8682-154fd0bfc32c admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:48.723303 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2342e-f6fd-4ad7-8682-154fd0bfc32c admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:48.723989 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2342e-f6fd-4ad7-8682-154fd0bfc32c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:48.725985 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2342e-f6fd-4ad7-8682-154fd0bfc32c admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:48.746585 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1387/2790] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:41:48 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:41:48.750628 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc86b3-dd23-4565-8df0-3ae9491ef536 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:48.760803 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc86b3-dd23-4565-8df0-3ae9491ef536 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:48.765720 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a6bc96-4f9c-4b07-96c2-043a2d73002f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:48.766392 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6bc96-4f9c-4b07-96c2-043a2d73002f admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:48.766683 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6bc96-4f9c-4b07-96c2-043a2d73002f admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:48.766993 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6bc96-4f9c-4b07-96c2-043a2d73002f admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:48.767719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6bc96-4f9c-4b07-96c2-043a2d73002f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:48.769345 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6bc96-4f9c-4b07-96c2-043a2d73002f admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:48.771473 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:48.785859 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1404/2791] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:41:48.790412 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2342e-f6fd-4ad7-8682-154fd0bfc32c admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:48.800360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2342e-f6fd-4ad7-8682-154fd0bfc32c admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:48.804964 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc96d79c-d716-4be4-9cbf-73845b2b733f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:48.806253 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96d79c-d716-4be4-9cbf-73845b2b733f admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:48.806537 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96d79c-d716-4be4-9cbf-73845b2b733f admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:48.806818 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96d79c-d716-4be4-9cbf-73845b2b733f admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:48.807574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc96d79c-d716-4be4-9cbf-73845b2b733f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:48.809201 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc96d79c-d716-4be4-9cbf-73845b2b733f admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:48.818972 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1388/2792] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:41:48.823033 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a6bc96-4f9c-4b07-96c2-043a2d73002f admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:48.833030 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a6bc96-4f9c-4b07-96c2-043a2d73002f admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:48.838511 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59fd86-fe7a-4f80-b09f-a3eecff43e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:48.839388 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59fd86-fe7a-4f80-b09f-a3eecff43e94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:48.839671 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59fd86-fe7a-4f80-b09f-a3eecff43e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:48.839947 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59fd86-fe7a-4f80-b09f-a3eecff43e94 admin admin] PATH_INFO: `/v3/projects/e80338867cd04f08be199ecc9f3993bb/users/7fcaaba2a9084fd29c3737ae3440eae9/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:48.854849 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b59fd86-fe7a-4f80-b09f-a3eecff43e94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e80338867cd04f08be199ecc9f3993bb, user_id=7fcaaba2a9084fd29c3737ae3440eae9, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:48.857529 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b59fd86-fe7a-4f80-b09f-a3eecff43e94 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:48.870639 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1405/2793] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:41:48 2022] PUT /identity/v3/projects/e80338867cd04f08be199ecc9f3993bb/users/7fcaaba2a9084fd29c3737ae3440eae9/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:48.880908 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922027a6-bacb-4525-863f-43436402f5e0 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:48.881144 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922027a6-bacb-4525-863f-43436402f5e0 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:48.881462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922027a6-bacb-4525-863f-43436402f5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:48.913809 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-922027a6-bacb-4525-863f-43436402f5e0 None None] MFA Rules not processed for user `7fcaaba2a9084fd29c3737ae3440eae9`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:48.915692 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-922027a6-bacb-4525-863f-43436402f5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:48.986412 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1389/2794] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:41:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:41:48.990843 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59fd86-fe7a-4f80-b09f-a3eecff43e94 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:49.053005 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59fd86-fe7a-4f80-b09f-a3eecff43e94 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:49.057860 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab9b00c-2fa0-4168-8988-b67b8581449f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:49.058516 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab9b00c-2fa0-4168-8988-b67b8581449f admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:49.058797 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab9b00c-2fa0-4168-8988-b67b8581449f admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:49.059071 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab9b00c-2fa0-4168-8988-b67b8581449f admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:49.059758 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab9b00c-2fa0-4168-8988-b67b8581449f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:49.061658 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab9b00c-2fa0-4168-8988-b67b8581449f admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:49.086053 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1406/2795] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:41:48 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:41:49.090909 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922027a6-bacb-4525-863f-43436402f5e0 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:49.101664 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922027a6-bacb-4525-863f-43436402f5e0 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:49.157774 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a209700-e0c7-4a53-98bc-a4c3ff10f5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:49.158369 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a209700-e0c7-4a53-98bc-a4c3ff10f5fc admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:49.158655 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a209700-e0c7-4a53-98bc-a4c3ff10f5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:49.158934 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a209700-e0c7-4a53-98bc-a4c3ff10f5fc admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:49.159617 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a209700-e0c7-4a53-98bc-a4c3ff10f5fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:49.161596 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a209700-e0c7-4a53-98bc-a4c3ff10f5fc admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:49.168715 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:49.183092 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1390/2796] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:41:49 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:41:49.187240 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab9b00c-2fa0-4168-8988-b67b8581449f admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:49.197656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab9b00c-2fa0-4168-8988-b67b8581449f admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:49.202513 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90724c6b-2770-4536-8a65-c3a39098ce7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:49.203179 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90724c6b-2770-4536-8a65-c3a39098ce7b admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:49.203468 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90724c6b-2770-4536-8a65-c3a39098ce7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:49.203753 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90724c6b-2770-4536-8a65-c3a39098ce7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:49.204774 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90724c6b-2770-4536-8a65-c3a39098ce7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:49.206408 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90724c6b-2770-4536-8a65-c3a39098ce7b admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:49.222301 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1407/2797] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:41:49.226272 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a209700-e0c7-4a53-98bc-a4c3ff10f5fc admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:49.237623 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a209700-e0c7-4a53-98bc-a4c3ff10f5fc admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:49.242515 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb89d2-152a-45c2-a70e-1fdccd0597bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:49.247381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb89d2-152a-45c2-a70e-1fdccd0597bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:49.247671 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb89d2-152a-45c2-a70e-1fdccd0597bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:49.247955 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb89d2-152a-45c2-a70e-1fdccd0597bb admin admin] PATH_INFO: `/v3/projects/9db25e54b2504695a524d93579b6d41c/users/e88e6d8dd26a4a2984072a7cf512e899/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:49.267889 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb89d2-152a-45c2-a70e-1fdccd0597bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9db25e54b2504695a524d93579b6d41c, user_id=e88e6d8dd26a4a2984072a7cf512e899, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:49.270829 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb89d2-152a-45c2-a70e-1fdccd0597bb admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:49.285346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1391/2798] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:41:49 2022] PUT /identity/v3/projects/9db25e54b2504695a524d93579b6d41c/users/e88e6d8dd26a4a2984072a7cf512e899/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:49.289252 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90724c6b-2770-4536-8a65-c3a39098ce7b admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:49.299703 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90724c6b-2770-4536-8a65-c3a39098ce7b admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:49.356504 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26cb48c-c720-48b4-93e0-c3d8d738a64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:49.357208 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cb48c-c720-48b4-93e0-c3d8d738a64a admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:49.357438 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cb48c-c720-48b4-93e0-c3d8d738a64a admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:49.357714 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cb48c-c720-48b4-93e0-c3d8d738a64a admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:49.358431 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26cb48c-c720-48b4-93e0-c3d8d738a64a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:49.360017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26cb48c-c720-48b4-93e0-c3d8d738a64a admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:49.370256 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1408/2799] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:41:49.374552 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb89d2-152a-45c2-a70e-1fdccd0597bb admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:49.385471 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb89d2-152a-45c2-a70e-1fdccd0597bb admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:49.390075 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ddc83-7854-438f-a1d7-a8ffc2fea666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:49.390959 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ddc83-7854-438f-a1d7-a8ffc2fea666 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:49.391246 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ddc83-7854-438f-a1d7-a8ffc2fea666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:49.391529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ddc83-7854-438f-a1d7-a8ffc2fea666 admin admin] PATH_INFO: `/v3/projects/9db25e54b2504695a524d93579b6d41c/users/e88e6d8dd26a4a2984072a7cf512e899/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:49.393527 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ddc83-7854-438f-a1d7-a8ffc2fea666 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9db25e54b2504695a524d93579b6d41c, user_id=e88e6d8dd26a4a2984072a7cf512e899, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:49.396149 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ddc83-7854-438f-a1d7-a8ffc2fea666 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:49.407087 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1392/2800] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:41:49 2022] PUT /identity/v3/projects/9db25e54b2504695a524d93579b6d41c/users/e88e6d8dd26a4a2984072a7cf512e899/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:49.417293 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412d51c-585f-4044-8c90-f013beb8da1b None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:49.417523 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412d51c-585f-4044-8c90-f013beb8da1b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:49.417807 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412d51c-585f-4044-8c90-f013beb8da1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:49.444017 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-4412d51c-585f-4044-8c90-f013beb8da1b None None] MFA Rules not processed for user `e88e6d8dd26a4a2984072a7cf512e899`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:49.445256 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-4412d51c-585f-4044-8c90-f013beb8da1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:49.478294 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:49.515585 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1409/2801] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:41:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:41:49.527723 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ddc83-7854-438f-a1d7-a8ffc2fea666 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:49.529620 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-171ddc83-7854-438f-a1d7-a8ffc2fea666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:49.593391 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ddc83-7854-438f-a1d7-a8ffc2fea666 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:49.599505 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45694580-85b4-45f4-bdd8-045cc0b997a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:49.600209 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45694580-85b4-45f4-bdd8-045cc0b997a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:49.600489 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45694580-85b4-45f4-bdd8-045cc0b997a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:49.600772 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45694580-85b4-45f4-bdd8-045cc0b997a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:49.664802 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45694580-85b4-45f4-bdd8-045cc0b997a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:49.666892 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45694580-85b4-45f4-bdd8-045cc0b997a6 service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:49.675912 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1393/2802] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:41:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:41:51.153316 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412d51c-585f-4044-8c90-f013beb8da1b None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:51.163263 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412d51c-585f-4044-8c90-f013beb8da1b None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:51.167529 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1100c85-903f-4f7d-8814-fa6307988aef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:51.168189 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1100c85-903f-4f7d-8814-fa6307988aef service cinder] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:51.168474 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1100c85-903f-4f7d-8814-fa6307988aef service cinder] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:51.168749 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1100c85-903f-4f7d-8814-fa6307988aef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:51.231027 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1100c85-903f-4f7d-8814-fa6307988aef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:51.232409 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1100c85-903f-4f7d-8814-fa6307988aef service cinder] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:51.242555 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1410/2803] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:41:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:41:51.809994 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45694580-85b4-45f4-bdd8-045cc0b997a6 service cinder] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:51.818483 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45694580-85b4-45f4-bdd8-045cc0b997a6 service cinder] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:51.822970 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb23cc-da5f-4ba3-8efa-8467cb3e0c53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:51.823604 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb23cc-da5f-4ba3-8efa-8467cb3e0c53 service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:51.823885 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb23cc-da5f-4ba3-8efa-8467cb3e0c53 service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:51.824162 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb23cc-da5f-4ba3-8efa-8467cb3e0c53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:51.833256 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddb23cc-da5f-4ba3-8efa-8467cb3e0c53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:51.834676 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddb23cc-da5f-4ba3-8efa-8467cb3e0c53 service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:51.844373 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1394/2804] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:41:51 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:41:58.834382 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1100c85-903f-4f7d-8814-fa6307988aef service cinder] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:58.896794 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1100c85-903f-4f7d-8814-fa6307988aef service cinder] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:58.902035 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b04dc-f328-4e1c-a73e-8c0518769e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:58.902894 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b04dc-f328-4e1c-a73e-8c0518769e33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:58.903173 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b04dc-f328-4e1c-a73e-8c0518769e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:58.903445 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b04dc-f328-4e1c-a73e-8c0518769e33 admin admin] PATH_INFO: `/v3/users/98c568af160746ea948dfe8c54b6fe6f` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:58.904731 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b04dc-f328-4e1c-a73e-8c0518769e33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98c568af160746ea948dfe8c54b6fe6f)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:58.906901 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b04dc-f328-4e1c-a73e-8c0518769e33 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:58.955799 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8e8b04dc-f328-4e1c-a73e-8c0518769e33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98c568af160746ea948dfe8c54b6fe6f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:58.968797 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8e8b04dc-f328-4e1c-a73e-8c0518769e33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98c568af160746ea948dfe8c54b6fe6f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:58.972533 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:58.976543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8e8b04dc-f328-4e1c-a73e-8c0518769e33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98c568af160746ea948dfe8c54b6fe6f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:58.977385 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8e8b04dc-f328-4e1c-a73e-8c0518769e33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98c568af160746ea948dfe8c54b6fe6f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:58.978538 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8e8b04dc-f328-4e1c-a73e-8c0518769e33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98c568af160746ea948dfe8c54b6fe6f'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:58.991578 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1411/2805] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:41:58 2022] DELETE /identity/v3/users/98c568af160746ea948dfe8c54b6fe6f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:58.996898 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28239f-1b1d-4f3c-95b9-f3b96576d4d9 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:58.997662 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28239f-1b1d-4f3c-95b9-f3b96576d4d9 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:58.997884 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28239f-1b1d-4f3c-95b9-f3b96576d4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:59.015202 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-4f28239f-1b1d-4f3c-95b9-f3b96576d4d9 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:41:59.016715 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-4f28239f-1b1d-4f3c-95b9-f3b96576d4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:59.068638 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:41:59.082208 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1395/2806] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:41:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:41:59.092057 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b04dc-f328-4e1c-a73e-8c0518769e33 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:59.093485 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-8e8b04dc-f328-4e1c-a73e-8c0518769e33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:59.149927 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b04dc-f328-4e1c-a73e-8c0518769e33 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:59.153814 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e592f79-ea30-4306-b5e2-87b7988d2730 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:59.154457 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e592f79-ea30-4306-b5e2-87b7988d2730 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:59.154740 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e592f79-ea30-4306-b5e2-87b7988d2730 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:59.155016 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e592f79-ea30-4306-b5e2-87b7988d2730 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:59.163787 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e592f79-ea30-4306-b5e2-87b7988d2730 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:59.165280 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e592f79-ea30-4306-b5e2-87b7988d2730 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:59.174977 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1412/2807] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:41:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:41:59.460088 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28239f-1b1d-4f3c-95b9-f3b96576d4d9 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:59.465580 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-4f28239f-1b1d-4f3c-95b9-f3b96576d4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:41:59.483021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28239f-1b1d-4f3c-95b9-f3b96576d4d9 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:59.487924 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96350b-f0c6-41f0-9776-6e7abe10eaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:59.488895 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96350b-f0c6-41f0-9776-6e7abe10eaa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:59.489319 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96350b-f0c6-41f0-9776-6e7abe10eaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:59.489741 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96350b-f0c6-41f0-9776-6e7abe10eaa7 admin admin] PATH_INFO: `/v3/projects/0cc7b8eb4a4a4312a4d104681ab640b4` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:59.491119 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96350b-f0c6-41f0-9776-6e7abe10eaa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cc7b8eb4a4a4312a4d104681ab640b4)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:59.493158 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96350b-f0c6-41f0-9776-6e7abe10eaa7 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:59.544624 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-db96350b-f0c6-41f0-9776-6e7abe10eaa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cc7b8eb4a4a4312a4d104681ab640b4', 'request_id': 'req-db96350b-f0c6-41f0-9776-6e7abe10eaa7'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:59.552339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-db96350b-f0c6-41f0-9776-6e7abe10eaa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cc7b8eb4a4a4312a4d104681ab640b4', 'request_id': 'req-db96350b-f0c6-41f0-9776-6e7abe10eaa7'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:59.559992 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-db96350b-f0c6-41f0-9776-6e7abe10eaa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cc7b8eb4a4a4312a4d104681ab640b4', 'request_id': 'req-db96350b-f0c6-41f0-9776-6e7abe10eaa7'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:59.571583 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-db96350b-f0c6-41f0-9776-6e7abe10eaa7 admin admin] The token cache is being invalidate because project 0cc7b8eb4a4a4312a4d104681ab640b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:41:59.571881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-db96350b-f0c6-41f0-9776-6e7abe10eaa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:59.572721 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-db96350b-f0c6-41f0-9776-6e7abe10eaa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:59.574410 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1396/2808] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:41:59 2022] DELETE /identity/v3/projects/0cc7b8eb4a4a4312a4d104681ab640b4 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:41:59.579008 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e592f79-ea30-4306-b5e2-87b7988d2730 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:41:59.589692 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e592f79-ea30-4306-b5e2-87b7988d2730 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:41:59.645916 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ff85b-4b4f-4e26-bb7e-b1f4a2ffec76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:41:59.646792 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ff85b-4b4f-4e26-bb7e-b1f4a2ffec76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:41:59.647076 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ff85b-4b4f-4e26-bb7e-b1f4a2ffec76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:41:59.647357 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ff85b-4b4f-4e26-bb7e-b1f4a2ffec76 admin admin] PATH_INFO: `/v3/users/7fcaaba2a9084fd29c3737ae3440eae9` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:41:59.648563 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839ff85b-4b4f-4e26-bb7e-b1f4a2ffec76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fcaaba2a9084fd29c3737ae3440eae9)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:41:59.650517 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839ff85b-4b4f-4e26-bb7e-b1f4a2ffec76 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:41:59.701244 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-839ff85b-4b4f-4e26-bb7e-b1f4a2ffec76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fcaaba2a9084fd29c3737ae3440eae9'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:59.714820 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-839ff85b-4b4f-4e26-bb7e-b1f4a2ffec76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fcaaba2a9084fd29c3737ae3440eae9'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:59.724163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-839ff85b-4b4f-4e26-bb7e-b1f4a2ffec76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fcaaba2a9084fd29c3737ae3440eae9'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:59.724800 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-839ff85b-4b4f-4e26-bb7e-b1f4a2ffec76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fcaaba2a9084fd29c3737ae3440eae9'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:59.725401 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-839ff85b-4b4f-4e26-bb7e-b1f4a2ffec76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fcaaba2a9084fd29c3737ae3440eae9'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:41:59.739250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1413/2809] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:41:59 2022] DELETE /identity/v3/users/7fcaaba2a9084fd29c3737ae3440eae9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:42:00.005543 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96350b-f0c6-41f0-9776-6e7abe10eaa7 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:42:00.006997 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-db96350b-f0c6-41f0-9776-6e7abe10eaa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:42:00.025055 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96350b-f0c6-41f0-9776-6e7abe10eaa7 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:42:00.100346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c527bfc-3109-4963-aff0-8a7285c602f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:42:00.101547 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c527bfc-3109-4963-aff0-8a7285c602f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:00.101839 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c527bfc-3109-4963-aff0-8a7285c602f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:00.102120 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c527bfc-3109-4963-aff0-8a7285c602f0 admin admin] PATH_INFO: `/v3/projects/e80338867cd04f08be199ecc9f3993bb` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:00.103357 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c527bfc-3109-4963-aff0-8a7285c602f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e80338867cd04f08be199ecc9f3993bb)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:42:00.105681 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c527bfc-3109-4963-aff0-8a7285c602f0 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:42:00.153645 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-0c527bfc-3109-4963-aff0-8a7285c602f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e80338867cd04f08be199ecc9f3993bb', 'request_id': 'req-0c527bfc-3109-4963-aff0-8a7285c602f0'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:42:00.160857 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-0c527bfc-3109-4963-aff0-8a7285c602f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e80338867cd04f08be199ecc9f3993bb', 'request_id': 'req-0c527bfc-3109-4963-aff0-8a7285c602f0'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:42:00.167494 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-0c527bfc-3109-4963-aff0-8a7285c602f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e80338867cd04f08be199ecc9f3993bb', 'request_id': 'req-0c527bfc-3109-4963-aff0-8a7285c602f0'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:42:00.178067 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-0c527bfc-3109-4963-aff0-8a7285c602f0 admin admin] The token cache is being invalidate because project e80338867cd04f08be199ecc9f3993bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:42:00.178397 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-0c527bfc-3109-4963-aff0-8a7285c602f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:42:00.179034 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-0c527bfc-3109-4963-aff0-8a7285c602f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:42:00.180606 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1397/2810] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:42:00 2022] DELETE /identity/v3/projects/e80338867cd04f08be199ecc9f3993bb => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:42:00.184941 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ff85b-4b4f-4e26-bb7e-b1f4a2ffec76 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:42:00.186336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-839ff85b-4b4f-4e26-bb7e-b1f4a2ffec76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:42:00.199221 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ff85b-4b4f-4e26-bb7e-b1f4a2ffec76 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:42:00.203723 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:42:00.259231 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb54f67-8d56-42cb-bc61-3b5fa23ad5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:42:00.260647 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb54f67-8d56-42cb-bc61-3b5fa23ad5a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:00.261034 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb54f67-8d56-42cb-bc61-3b5fa23ad5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:00.261333 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb54f67-8d56-42cb-bc61-3b5fa23ad5a4 admin admin] PATH_INFO: `/v3/users/e88e6d8dd26a4a2984072a7cf512e899` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:00.262543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb54f67-8d56-42cb-bc61-3b5fa23ad5a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e88e6d8dd26a4a2984072a7cf512e899)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:42:00.264522 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb54f67-8d56-42cb-bc61-3b5fa23ad5a4 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:42:00.313231 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8cb54f67-8d56-42cb-bc61-3b5fa23ad5a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e88e6d8dd26a4a2984072a7cf512e899'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:42:00.326388 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8cb54f67-8d56-42cb-bc61-3b5fa23ad5a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e88e6d8dd26a4a2984072a7cf512e899'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:42:00.333995 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8cb54f67-8d56-42cb-bc61-3b5fa23ad5a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e88e6d8dd26a4a2984072a7cf512e899'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:42:00.334719 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8cb54f67-8d56-42cb-bc61-3b5fa23ad5a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e88e6d8dd26a4a2984072a7cf512e899'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:42:00.335394 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-8cb54f67-8d56-42cb-bc61-3b5fa23ad5a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e88e6d8dd26a4a2984072a7cf512e899'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:42:00.348274 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1414/2811] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:42:00 2022] DELETE /identity/v3/users/e88e6d8dd26a4a2984072a7cf512e899 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:42:00.628006 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c527bfc-3109-4963-aff0-8a7285c602f0 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:42:00.629529 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-0c527bfc-3109-4963-aff0-8a7285c602f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:42:00.645198 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c527bfc-3109-4963-aff0-8a7285c602f0 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:42:00.649652 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:42:00.707105 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b40ef4-a036-4b10-8809-d76e5de7fb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:42:00.707888 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b40ef4-a036-4b10-8809-d76e5de7fb26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:00.708192 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b40ef4-a036-4b10-8809-d76e5de7fb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:00.708479 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b40ef4-a036-4b10-8809-d76e5de7fb26 admin admin] PATH_INFO: `/v3/projects/9db25e54b2504695a524d93579b6d41c` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:00.709896 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b40ef4-a036-4b10-8809-d76e5de7fb26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9db25e54b2504695a524d93579b6d41c)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:42:00.711921 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b40ef4-a036-4b10-8809-d76e5de7fb26 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:42:00.767706 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-c6b40ef4-a036-4b10-8809-d76e5de7fb26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9db25e54b2504695a524d93579b6d41c', 'request_id': 'req-c6b40ef4-a036-4b10-8809-d76e5de7fb26'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:42:00.775199 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-c6b40ef4-a036-4b10-8809-d76e5de7fb26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9db25e54b2504695a524d93579b6d41c', 'request_id': 'req-c6b40ef4-a036-4b10-8809-d76e5de7fb26'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:42:00.781556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-c6b40ef4-a036-4b10-8809-d76e5de7fb26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9db25e54b2504695a524d93579b6d41c', 'request_id': 'req-c6b40ef4-a036-4b10-8809-d76e5de7fb26'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:42:00.791702 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-c6b40ef4-a036-4b10-8809-d76e5de7fb26 admin admin] The token cache is being invalidate because project 9db25e54b2504695a524d93579b6d41c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:42:00.792008 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-c6b40ef4-a036-4b10-8809-d76e5de7fb26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:42:00.792648 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-c6b40ef4-a036-4b10-8809-d76e5de7fb26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:42:00.794198 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1398/2812] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:42:00 2022] DELETE /identity/v3/projects/9db25e54b2504695a524d93579b6d41c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:42:00.800258 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38da11-60d5-44b6-8b9a-832b5f5521f1 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:00.800552 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38da11-60d5-44b6-8b9a-832b5f5521f1 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:00.800839 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38da11-60d5-44b6-8b9a-832b5f5521f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:00.819158 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-1e38da11-60d5-44b6-8b9a-832b5f5521f1 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:42:00.820391 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1e38da11-60d5-44b6-8b9a-832b5f5521f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:42:00.882385 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1415/2813] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:42:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:42:00.891144 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3e500-0a65-4659-bd99-69dc07b43ed3 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:00.891482 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3e500-0a65-4659-bd99-69dc07b43ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:00.891810 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3e500-0a65-4659-bd99-69dc07b43ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:00.911955 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-f6f3e500-0a65-4659-bd99-69dc07b43ed3 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:42:00.913178 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-f6f3e500-0a65-4659-bd99-69dc07b43ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:42:00.922038 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1399/2814] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:42:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:42:00.926539 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38da11-60d5-44b6-8b9a-832b5f5521f1 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:42:00.937886 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38da11-60d5-44b6-8b9a-832b5f5521f1 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:42:00.942448 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccea7b3-6bb9-44a1-8529-de39f30b8c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:42:00.943101 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccea7b3-6bb9-44a1-8529-de39f30b8c2a admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:00.943393 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccea7b3-6bb9-44a1-8529-de39f30b8c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:00.943677 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccea7b3-6bb9-44a1-8529-de39f30b8c2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:00.944437 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccea7b3-6bb9-44a1-8529-de39f30b8c2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:42:00.944751 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccea7b3-6bb9-44a1-8529-de39f30b8c2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:42:00.946360 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccea7b3-6bb9-44a1-8529-de39f30b8c2a admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:42:00.958232 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1416/2815] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:42:00.962450 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3e500-0a65-4659-bd99-69dc07b43ed3 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:42:00.972060 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3e500-0a65-4659-bd99-69dc07b43ed3 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:42:00.976694 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4dd90b-6441-4816-984b-a6c9e34ee539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:42:00.977357 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4dd90b-6441-4816-984b-a6c9e34ee539 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:00.977644 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4dd90b-6441-4816-984b-a6c9e34ee539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:00.977923 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4dd90b-6441-4816-984b-a6c9e34ee539 admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:00.979371 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4dd90b-6441-4816-984b-a6c9e34ee539 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:42:00.981415 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4dd90b-6441-4816-984b-a6c9e34ee539 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:42:01.006893 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1400/2816] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:42:00 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:42:01.011326 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccea7b3-6bb9-44a1-8529-de39f30b8c2a admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:42:01.021847 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccea7b3-6bb9-44a1-8529-de39f30b8c2a admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:42:01.033882 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:42:01.083953 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfbe8b-0100-47a4-b24a-d159100695d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:42:01.084680 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfbe8b-0100-47a4-b24a-d159100695d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:01.085119 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfbe8b-0100-47a4-b24a-d159100695d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:01.085405 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfbe8b-0100-47a4-b24a-d159100695d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:01.086128 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dfbe8b-0100-47a4-b24a-d159100695d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:42:01.088360 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dfbe8b-0100-47a4-b24a-d159100695d1 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:42:01.108154 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1417/2817] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:42:01 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:42:01.112259 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4dd90b-6441-4816-984b-a6c9e34ee539 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:42:01.123232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4dd90b-6441-4816-984b-a6c9e34ee539 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:42:01.127834 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edbb4f3-a22a-4327-ac66-e22502532f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:42:01.128515 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbb4f3-a22a-4327-ac66-e22502532f92 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:01.128831 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbb4f3-a22a-4327-ac66-e22502532f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:01.129111 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbb4f3-a22a-4327-ac66-e22502532f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:01.129841 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edbb4f3-a22a-4327-ac66-e22502532f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:42:01.131464 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edbb4f3-a22a-4327-ac66-e22502532f92 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:42:01.141891 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1401/2818] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:42:01.145746 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfbe8b-0100-47a4-b24a-d159100695d1 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:42:01.155730 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfbe8b-0100-47a4-b24a-d159100695d1 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:42:01.160275 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98b3dce-3c46-4fe8-a3b9-8169315a9d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:42:01.160925 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b3dce-3c46-4fe8-a3b9-8169315a9d12 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:01.161206 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b3dce-3c46-4fe8-a3b9-8169315a9d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:01.161482 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b3dce-3c46-4fe8-a3b9-8169315a9d12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:01.162195 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98b3dce-3c46-4fe8-a3b9-8169315a9d12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:42:01.163763 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98b3dce-3c46-4fe8-a3b9-8169315a9d12 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:42:01.174666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1418/2819] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:42:01.179609 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edbb4f3-a22a-4327-ac66-e22502532f92 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:42:01.195039 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edbb4f3-a22a-4327-ac66-e22502532f92 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:42:01.202167 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45176d1-66dd-4d53-b66a-10a52aa0b1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:42:01.203544 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45176d1-66dd-4d53-b66a-10a52aa0b1fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:01.203947 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45176d1-66dd-4d53-b66a-10a52aa0b1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:01.204318 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45176d1-66dd-4d53-b66a-10a52aa0b1fa admin admin] PATH_INFO: `/v3/projects/d05c15fec3294719b7f8bc5690f355f9/users/c7c4f8f2298d4014b2a08aaa391751f9/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:01.220739 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45176d1-66dd-4d53-b66a-10a52aa0b1fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d05c15fec3294719b7f8bc5690f355f9, user_id=c7c4f8f2298d4014b2a08aaa391751f9, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:42:01.223594 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45176d1-66dd-4d53-b66a-10a52aa0b1fa admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:42:01.237880 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1402/2820] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:42:01 2022] PUT /identity/v3/projects/d05c15fec3294719b7f8bc5690f355f9/users/c7c4f8f2298d4014b2a08aaa391751f9/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:42:01.243099 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08668e0d-05ff-41d3-9705-081561477982 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:01.243570 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08668e0d-05ff-41d3-9705-081561477982 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:01.243685 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08668e0d-05ff-41d3-9705-081561477982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:01.261666 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-08668e0d-05ff-41d3-9705-081561477982 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:42:01.263019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-08668e0d-05ff-41d3-9705-081561477982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:42:01.322426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1419/2821] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:42:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:42:01.332297 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45176d1-66dd-4d53-b66a-10a52aa0b1fa admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:42:01.334021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-a45176d1-66dd-4d53-b66a-10a52aa0b1fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:42:01.370709 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:42:01.393828 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45176d1-66dd-4d53-b66a-10a52aa0b1fa admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:42:01.397569 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a50eb-16d9-4c5c-af71-45874bfc4a69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:42:01.398234 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a50eb-16d9-4c5c-af71-45874bfc4a69 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:01.398524 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a50eb-16d9-4c5c-af71-45874bfc4a69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:01.398811 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a50eb-16d9-4c5c-af71-45874bfc4a69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:01.410266 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a50eb-16d9-4c5c-af71-45874bfc4a69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:42:01.412768 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a50eb-16d9-4c5c-af71-45874bfc4a69 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:42:01.424601 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1403/2822] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:42:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:42:02.075797 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bce73-78b0-4283-8dda-b7ab83188178 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:02.076336 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bce73-78b0-4283-8dda-b7ab83188178 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:02.076842 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bce73-78b0-4283-8dda-b7ab83188178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:02.111240 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-5d4bce73-78b0-4283-8dda-b7ab83188178 None None] MFA Rules not processed for user `c7c4f8f2298d4014b2a08aaa391751f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:42:02.112486 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-5d4bce73-78b0-4283-8dda-b7ab83188178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:42:02.186747 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1420/2823] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:42:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:42:02.195416 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a50eb-16d9-4c5c-af71-45874bfc4a69 service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:42:02.197286 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-c99a50eb-16d9-4c5c-af71-45874bfc4a69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:42:02.264316 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a50eb-16d9-4c5c-af71-45874bfc4a69 service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:42:02.268083 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da734e2a-bfa8-4ed3-94b2-1e75781d80ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e12ef3e115544a51a6bf9b08d265fefb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:42:02.268763 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da734e2a-bfa8-4ed3-94b2-1e75781d80ed service cinder] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:02.269056 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da734e2a-bfa8-4ed3-94b2-1e75781d80ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:02.269339 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da734e2a-bfa8-4ed3-94b2-1e75781d80ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:02.278436 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da734e2a-bfa8-4ed3-94b2-1e75781d80ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:42:02.279879 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da734e2a-bfa8-4ed3-94b2-1e75781d80ed service cinder] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:42:02.288698 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1404/2824] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:42:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:42:03.699797 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3de87-ecaa-421e-89a9-1386f67e46b4 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:03.700407 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3de87-ecaa-421e-89a9-1386f67e46b4 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:03.700407 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3de87-ecaa-421e-89a9-1386f67e46b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:03.719674 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-d7a3de87-ecaa-421e-89a9-1386f67e46b4 None None] MFA Rules not processed for user `c7c4f8f2298d4014b2a08aaa391751f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:42:03.720914 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d7a3de87-ecaa-421e-89a9-1386f67e46b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:42:03.728193 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1421/2825] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:42:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:42:03.974719 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da734e2a-bfa8-4ed3-94b2-1e75781d80ed service cinder] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:42:03.976226 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-da734e2a-bfa8-4ed3-94b2-1e75781d80ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:42:03.989727 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da734e2a-bfa8-4ed3-94b2-1e75781d80ed service cinder] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:42:03.993926 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13747b-c8f9-4479-be1a-3b991097832e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:42:03.994574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13747b-c8f9-4479-be1a-3b991097832e service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:03.994856 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13747b-c8f9-4479-be1a-3b991097832e service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:03.995135 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13747b-c8f9-4479-be1a-3b991097832e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:03.996903 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-7f13747b-c8f9-4479-be1a-3b991097832e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:42:04.006787 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f13747b-c8f9-4479-be1a-3b991097832e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:42:04.008337 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f13747b-c8f9-4479-be1a-3b991097832e service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:42:04.017341 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1405/2826] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:42:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:42:07.134119 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a3de87-ecaa-421e-89a9-1386f67e46b4 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:42:07.135722 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-d7a3de87-ecaa-421e-89a9-1386f67e46b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:42:07.149796 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a3de87-ecaa-421e-89a9-1386f67e46b4 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:42:07.154361 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cff3b6-2c87-49f8-86fe-a03ede054bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:42:07.155281 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cff3b6-2c87-49f8-86fe-a03ede054bbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:07.155565 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cff3b6-2c87-49f8-86fe-a03ede054bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:07.155848 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cff3b6-2c87-49f8-86fe-a03ede054bbc admin admin] PATH_INFO: `/v3/users/b6f6c6a6728b4a9bb00d6f7eaae715d7` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:07.157062 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cff3b6-2c87-49f8-86fe-a03ede054bbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6f6c6a6728b4a9bb00d6f7eaae715d7)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:42:07.158952 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cff3b6-2c87-49f8-86fe-a03ede054bbc admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:42:07.172044 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:42:07.206406 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-71cff3b6-2c87-49f8-86fe-a03ede054bbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6f6c6a6728b4a9bb00d6f7eaae715d7'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:42:07.218855 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-71cff3b6-2c87-49f8-86fe-a03ede054bbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6f6c6a6728b4a9bb00d6f7eaae715d7'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:42:07.225999 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-71cff3b6-2c87-49f8-86fe-a03ede054bbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6f6c6a6728b4a9bb00d6f7eaae715d7'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:42:07.226659 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-71cff3b6-2c87-49f8-86fe-a03ede054bbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6f6c6a6728b4a9bb00d6f7eaae715d7'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:42:07.227264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-71cff3b6-2c87-49f8-86fe-a03ede054bbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6f6c6a6728b4a9bb00d6f7eaae715d7'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:42:07.240238 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1422/2827] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:42:07 2022] DELETE /identity/v3/users/b6f6c6a6728b4a9bb00d6f7eaae715d7 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:42:07.245634 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e99b8-6194-4c2e-9105-a97606f5022d None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:07.246169 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e99b8-6194-4c2e-9105-a97606f5022d None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:07.246281 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e99b8-6194-4c2e-9105-a97606f5022d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:07.264081 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-7e2e99b8-6194-4c2e-9105-a97606f5022d None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:42:07.265316 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-7e2e99b8-6194-4c2e-9105-a97606f5022d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:42:07.326482 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1406/2828] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:42:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:42:07.336308 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cff3b6-2c87-49f8-86fe-a03ede054bbc admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:42:07.337800 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-71cff3b6-2c87-49f8-86fe-a03ede054bbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:42:07.394405 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cff3b6-2c87-49f8-86fe-a03ede054bbc admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:42:07.398282 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78656357-de83-427a-9eb8-0ad8f831c607 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:42:07.399036 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78656357-de83-427a-9eb8-0ad8f831c607 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:07.399327 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78656357-de83-427a-9eb8-0ad8f831c607 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:07.399607 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78656357-de83-427a-9eb8-0ad8f831c607 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:07.408913 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78656357-de83-427a-9eb8-0ad8f831c607 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:42:07.410359 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78656357-de83-427a-9eb8-0ad8f831c607 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:42:07.420128 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1423/2829] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:42:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:42:07.694285 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e99b8-6194-4c2e-9105-a97606f5022d None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:42:07.695838 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-7e2e99b8-6194-4c2e-9105-a97606f5022d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:42:07.714397 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e99b8-6194-4c2e-9105-a97606f5022d None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:42:07.719685 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d8eca-c18a-4ab3-a6ff-62d8c91490d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:42:07.720633 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d8eca-c18a-4ab3-a6ff-62d8c91490d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:07.720900 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d8eca-c18a-4ab3-a6ff-62d8c91490d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:07.721222 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d8eca-c18a-4ab3-a6ff-62d8c91490d1 admin admin] PATH_INFO: `/v3/projects/66d7a2d57eec445ca7f2d95a15f28b8b` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:07.722346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d8eca-c18a-4ab3-a6ff-62d8c91490d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66d7a2d57eec445ca7f2d95a15f28b8b)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:42:07.724326 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d8eca-c18a-4ab3-a6ff-62d8c91490d1 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:42:07.771760 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-153d8eca-c18a-4ab3-a6ff-62d8c91490d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66d7a2d57eec445ca7f2d95a15f28b8b', 'request_id': 'req-153d8eca-c18a-4ab3-a6ff-62d8c91490d1'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:42:07.778525 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-153d8eca-c18a-4ab3-a6ff-62d8c91490d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66d7a2d57eec445ca7f2d95a15f28b8b', 'request_id': 'req-153d8eca-c18a-4ab3-a6ff-62d8c91490d1'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:42:07.785370 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-153d8eca-c18a-4ab3-a6ff-62d8c91490d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66d7a2d57eec445ca7f2d95a15f28b8b', 'request_id': 'req-153d8eca-c18a-4ab3-a6ff-62d8c91490d1'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:42:07.786810 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:42:07.799463 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-153d8eca-c18a-4ab3-a6ff-62d8c91490d1 admin admin] The token cache is being invalidate because project 66d7a2d57eec445ca7f2d95a15f28b8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:42:07.800422 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-153d8eca-c18a-4ab3-a6ff-62d8c91490d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:42:07.800839 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-153d8eca-c18a-4ab3-a6ff-62d8c91490d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:42:07.803345 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1407/2830] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:42:07 2022] DELETE /identity/v3/projects/66d7a2d57eec445ca7f2d95a15f28b8b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:42:07.808740 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949a21b-76e7-4a39-b2f4-6fe48b347ace None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:07.809143 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949a21b-76e7-4a39-b2f4-6fe48b347ace None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:07.809494 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949a21b-76e7-4a39-b2f4-6fe48b347ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:07.835506 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-2949a21b-76e7-4a39-b2f4-6fe48b347ace None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:42:07.836784 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-2949a21b-76e7-4a39-b2f4-6fe48b347ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:42:07.903584 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1424/2831] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:42:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:42:07.914161 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a3c8e-015c-44f0-b68b-1eb1ccca8c88 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:07.914530 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a3c8e-015c-44f0-b68b-1eb1ccca8c88 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:07.914866 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a3c8e-015c-44f0-b68b-1eb1ccca8c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:07.936472 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-3e1a3c8e-015c-44f0-b68b-1eb1ccca8c88 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:42:07.937705 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-3e1a3c8e-015c-44f0-b68b-1eb1ccca8c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:42:07.946148 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1408/2832] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:42:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 12:42:07.950849 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949a21b-76e7-4a39-b2f4-6fe48b347ace None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:42:07.961476 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949a21b-76e7-4a39-b2f4-6fe48b347ace None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:42:07.967417 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a32b1aa-eac4-4391-9430-497396464f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:42:07.968092 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32b1aa-eac4-4391-9430-497396464f63 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:07.968410 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32b1aa-eac4-4391-9430-497396464f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:07.968693 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32b1aa-eac4-4391-9430-497396464f63 admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:07.969426 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a32b1aa-eac4-4391-9430-497396464f63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:42:07.969738 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a32b1aa-eac4-4391-9430-497396464f63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:42:07.971367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a32b1aa-eac4-4391-9430-497396464f63 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:42:07.988092 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1425/2833] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 12:42:07.989297 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a3c8e-015c-44f0-b68b-1eb1ccca8c88 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:42:07.998792 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a3c8e-015c-44f0-b68b-1eb1ccca8c88 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:42:08.003468 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2185d2-8c5d-49c4-8669-9d8bbd26a77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:42:08.004119 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2185d2-8c5d-49c4-8669-9d8bbd26a77f admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:08.004439 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2185d2-8c5d-49c4-8669-9d8bbd26a77f admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:08.004723 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2185d2-8c5d-49c4-8669-9d8bbd26a77f admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:08.005422 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2185d2-8c5d-49c4-8669-9d8bbd26a77f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:42:08.007346 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2185d2-8c5d-49c4-8669-9d8bbd26a77f admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:42:08.038396 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1409/2834] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:42:07 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:42:08.042953 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a32b1aa-eac4-4391-9430-497396464f63 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:42:08.054616 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a32b1aa-eac4-4391-9430-497396464f63 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:42:08.074175 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:42:08.128142 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e0a9cc-02b6-471c-a797-78f9c11c9e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:42:08.128881 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0a9cc-02b6-471c-a797-78f9c11c9e39 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:08.129165 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0a9cc-02b6-471c-a797-78f9c11c9e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:08.129442 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0a9cc-02b6-471c-a797-78f9c11c9e39 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:08.130125 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e0a9cc-02b6-471c-a797-78f9c11c9e39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:42:08.132432 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e0a9cc-02b6-471c-a797-78f9c11c9e39 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:42:08.156105 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1426/2835] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:42:08 2022] POST /identity/v3/users => generated 290 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:42:08.160359 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2185d2-8c5d-49c4-8669-9d8bbd26a77f admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:42:08.173307 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2185d2-8c5d-49c4-8669-9d8bbd26a77f admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:42:08.180265 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d1fc7-f28c-469e-bd72-f2c98813ce5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:42:08.180941 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d1fc7-f28c-469e-bd72-f2c98813ce5d admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:08.181296 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d1fc7-f28c-469e-bd72-f2c98813ce5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:08.181580 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d1fc7-f28c-469e-bd72-f2c98813ce5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:08.182371 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d1fc7-f28c-469e-bd72-f2c98813ce5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:42:08.183979 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d1fc7-f28c-469e-bd72-f2c98813ce5d admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:42:08.201931 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1410/2836] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:42:08.206307 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e0a9cc-02b6-471c-a797-78f9c11c9e39 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:42:08.217163 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e0a9cc-02b6-471c-a797-78f9c11c9e39 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:42:08.221745 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e9d2c5-7a71-498d-92ed-9121df1ef79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:42:08.222400 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9d2c5-7a71-498d-92ed-9121df1ef79a admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:08.222686 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9d2c5-7a71-498d-92ed-9121df1ef79a admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:08.222967 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9d2c5-7a71-498d-92ed-9121df1ef79a admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:08.223688 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e9d2c5-7a71-498d-92ed-9121df1ef79a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:42:08.225325 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e9d2c5-7a71-498d-92ed-9121df1ef79a admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:42:08.236129 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1427/2837] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:42:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:42:08.240261 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d1fc7-f28c-469e-bd72-f2c98813ce5d admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:42:08.250340 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d1fc7-f28c-469e-bd72-f2c98813ce5d admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:42:08.255013 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c5399-5315-4408-9452-b6333188df1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:42:08.255882 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c5399-5315-4408-9452-b6333188df1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:08.256162 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c5399-5315-4408-9452-b6333188df1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:08.256462 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c5399-5315-4408-9452-b6333188df1a admin admin] PATH_INFO: `/v3/projects/ac5975413f5548c3881ec3c868295836/users/024cf85a89d445ff854fede06d0f9244/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:08.270902 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46c5399-5315-4408-9452-b6333188df1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac5975413f5548c3881ec3c868295836, user_id=024cf85a89d445ff854fede06d0f9244, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:42:08.273640 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46c5399-5315-4408-9452-b6333188df1a admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:42:08.289201 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1411/2838] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:42:08 2022] PUT /identity/v3/projects/ac5975413f5548c3881ec3c868295836/users/024cf85a89d445ff854fede06d0f9244/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:42:08.294932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1338bb4-22de-4bef-be00-25add17adb43 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:08.295226 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1338bb4-22de-4bef-be00-25add17adb43 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:08.295506 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1338bb4-22de-4bef-be00-25add17adb43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:08.313270 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-c1338bb4-22de-4bef-be00-25add17adb43 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:42:08.314507 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-c1338bb4-22de-4bef-be00-25add17adb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:42:08.377774 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1428/2839] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:42:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:42:08.388740 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c5399-5315-4408-9452-b6333188df1a admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:42:08.390566 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-d46c5399-5315-4408-9452-b6333188df1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:42:08.449231 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c5399-5315-4408-9452-b6333188df1a admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:42:08.452923 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d31a5-c7f6-462b-9f2e-4d20930d577b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:42:08.453572 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d31a5-c7f6-462b-9f2e-4d20930d577b service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:08.453853 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d31a5-c7f6-462b-9f2e-4d20930d577b service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:08.454133 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d31a5-c7f6-462b-9f2e-4d20930d577b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:08.464628 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415d31a5-c7f6-462b-9f2e-4d20930d577b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:42:08.466104 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415d31a5-c7f6-462b-9f2e-4d20930d577b service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:42:08.476196 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1412/2840] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:42:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:42:12.832661 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ab64a-fa6d-42fa-aa1f-c7b5db8b4a2d None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:12.833569 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ab64a-fa6d-42fa-aa1f-c7b5db8b4a2d None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:12.833926 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ab64a-fa6d-42fa-aa1f-c7b5db8b4a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:12.864153 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-3c7ab64a-fa6d-42fa-aa1f-c7b5db8b4a2d None None] MFA Rules not processed for user `024cf85a89d445ff854fede06d0f9244`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:42:12.865636 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3c7ab64a-fa6d-42fa-aa1f-c7b5db8b4a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:42:12.949492 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1429/2841] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:42:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:42:12.953733 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d31a5-c7f6-462b-9f2e-4d20930d577b service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:42:12.965205 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d31a5-c7f6-462b-9f2e-4d20930d577b service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:42:12.970031 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe243ba-e8ae-499e-bb15-a5ca49b7709c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:42:12.973638 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe243ba-e8ae-499e-bb15-a5ca49b7709c admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:12.974003 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe243ba-e8ae-499e-bb15-a5ca49b7709c admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:12.974782 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe243ba-e8ae-499e-bb15-a5ca49b7709c admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:12.975571 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe243ba-e8ae-499e-bb15-a5ca49b7709c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:42:12.977579 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe243ba-e8ae-499e-bb15-a5ca49b7709c admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:42:13.012718 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1413/2842] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:42:12 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:42:13.013987 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ab64a-fa6d-42fa-aa1f-c7b5db8b4a2d None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:42:13.025871 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7ab64a-fa6d-42fa-aa1f-c7b5db8b4a2d None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:42:13.052914 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:42:13.092381 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b2b20-2721-45fe-ada6-ae624f4cf50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:42:13.093268 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b2b20-2721-45fe-ada6-ae624f4cf50f admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:13.093698 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b2b20-2721-45fe-ada6-ae624f4cf50f admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:13.093984 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b2b20-2721-45fe-ada6-ae624f4cf50f admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:13.094680 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6b2b20-2721-45fe-ada6-ae624f4cf50f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:42:13.096630 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6b2b20-2721-45fe-ada6-ae624f4cf50f admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:42:13.118461 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1430/2843] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:42:13 2022] POST /identity/v3/users => generated 295 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:42:13.123369 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe243ba-e8ae-499e-bb15-a5ca49b7709c admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:42:13.133760 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe243ba-e8ae-499e-bb15-a5ca49b7709c admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:42:13.138850 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd11cf-df01-4db8-b8f1-922cd6b25a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:42:13.139512 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd11cf-df01-4db8-b8f1-922cd6b25a50 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:13.139797 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd11cf-df01-4db8-b8f1-922cd6b25a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:13.140076 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd11cf-df01-4db8-b8f1-922cd6b25a50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:13.140830 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd11cf-df01-4db8-b8f1-922cd6b25a50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:42:13.142428 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd11cf-df01-4db8-b8f1-922cd6b25a50 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:42:13.154620 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1414/2844] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:42:13.158720 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b2b20-2721-45fe-ada6-ae624f4cf50f admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:42:13.177341 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b2b20-2721-45fe-ada6-ae624f4cf50f admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:42:13.183016 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d93bdf-be49-4978-a0c4-a55f847e8538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:42:13.184199 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d93bdf-be49-4978-a0c4-a55f847e8538 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:13.184640 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d93bdf-be49-4978-a0c4-a55f847e8538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:13.185092 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d93bdf-be49-4978-a0c4-a55f847e8538 admin admin] PATH_INFO: `/v3/projects/2fbc1181e1654578946cfd81c1fdbb3c/users/3ad76b4b3d914432bb0a4cf2d9fdcd72/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:13.203067 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d93bdf-be49-4978-a0c4-a55f847e8538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fbc1181e1654578946cfd81c1fdbb3c, user_id=3ad76b4b3d914432bb0a4cf2d9fdcd72, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:42:13.206085 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d93bdf-be49-4978-a0c4-a55f847e8538 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:42:13.220070 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1431/2845] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:42:13 2022] PUT /identity/v3/projects/2fbc1181e1654578946cfd81c1fdbb3c/users/3ad76b4b3d914432bb0a4cf2d9fdcd72/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:42:13.223993 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd11cf-df01-4db8-b8f1-922cd6b25a50 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:42:13.234893 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd11cf-df01-4db8-b8f1-922cd6b25a50 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:42:13.289957 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:42:13.299448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081909d-022c-4715-a02f-0288bf2bcabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:42:13.300280 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081909d-022c-4715-a02f-0288bf2bcabf admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:13.300580 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081909d-022c-4715-a02f-0288bf2bcabf admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:13.301067 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081909d-022c-4715-a02f-0288bf2bcabf admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:13.302016 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081909d-022c-4715-a02f-0288bf2bcabf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:42:13.303937 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081909d-022c-4715-a02f-0288bf2bcabf admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:42:13.314406 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1415/2846] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:42:13.318537 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d93bdf-be49-4978-a0c4-a55f847e8538 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:42:13.332920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d93bdf-be49-4978-a0c4-a55f847e8538 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:42:13.340608 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb58686-0e5f-4357-b039-8ce7182f84c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:42:13.342735 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb58686-0e5f-4357-b039-8ce7182f84c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:13.343277 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb58686-0e5f-4357-b039-8ce7182f84c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:13.343277 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb58686-0e5f-4357-b039-8ce7182f84c6 admin admin] PATH_INFO: `/v3/projects/2fbc1181e1654578946cfd81c1fdbb3c/users/3ad76b4b3d914432bb0a4cf2d9fdcd72/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:13.349656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb58686-0e5f-4357-b039-8ce7182f84c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fbc1181e1654578946cfd81c1fdbb3c, user_id=3ad76b4b3d914432bb0a4cf2d9fdcd72, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:42:13.354612 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb58686-0e5f-4357-b039-8ce7182f84c6 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:42:13.368650 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1432/2847] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:42:13 2022] PUT /identity/v3/projects/2fbc1181e1654578946cfd81c1fdbb3c/users/3ad76b4b3d914432bb0a4cf2d9fdcd72/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:42:18.325681 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13acaa7f-a2ea-4270-8322-e9cb9d0bb1d8 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:18.326185 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13acaa7f-a2ea-4270-8322-e9cb9d0bb1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:18.326185 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13acaa7f-a2ea-4270-8322-e9cb9d0bb1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:18.366993 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-13acaa7f-a2ea-4270-8322-e9cb9d0bb1d8 None None] MFA Rules not processed for user `3ad76b4b3d914432bb0a4cf2d9fdcd72`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:42:18.368300 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-13acaa7f-a2ea-4270-8322-e9cb9d0bb1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:42:18.449377 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1416/2848] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:42:18 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:42:18.465502 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb58686-0e5f-4357-b039-8ce7182f84c6 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:42:18.467326 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-4eb58686-0e5f-4357-b039-8ce7182f84c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:42:18.545505 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb58686-0e5f-4357-b039-8ce7182f84c6 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:42:18.552383 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59498db3-8763-42bf-8322-6bb832ee3d6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:42:18.553132 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59498db3-8763-42bf-8322-6bb832ee3d6f service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:18.553326 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59498db3-8763-42bf-8322-6bb832ee3d6f service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:18.553753 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59498db3-8763-42bf-8322-6bb832ee3d6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:18.661057 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59498db3-8763-42bf-8322-6bb832ee3d6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:42:18.664361 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59498db3-8763-42bf-8322-6bb832ee3d6f service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:42:18.678285 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1433/2849] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:42:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:42:28.445865 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=38 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:42:28.674046 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=87,keystone:INSERT=5 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:42:38.552467 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7fc9f8-4a10-479e-864b-40265b0098e1 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:38.553058 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7fc9f8-4a10-479e-864b-40265b0098e1 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:38.553058 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7fc9f8-4a10-479e-864b-40265b0098e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:38.571168 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-6a7fc9f8-4a10-479e-864b-40265b0098e1 None None] MFA Rules not processed for user `024cf85a89d445ff854fede06d0f9244`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:42:38.572400 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-6a7fc9f8-4a10-479e-864b-40265b0098e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:42:38.580860 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1417/2850] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:42:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:42:48.565923 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:42:59.508492 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59498db3-8763-42bf-8322-6bb832ee3d6f service nova] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:42:59.509969 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-59498db3-8763-42bf-8322-6bb832ee3d6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:42:59.571561 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59498db3-8763-42bf-8322-6bb832ee3d6f service nova] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:42:59.575138 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d004507-a67f-4083-a6a1-a7389d66a99b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:42:59.575836 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d004507-a67f-4083-a6a1-a7389d66a99b service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:42:59.576098 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d004507-a67f-4083-a6a1-a7389d66a99b service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:42:59.576402 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d004507-a67f-4083-a6a1-a7389d66a99b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:42:59.586795 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d004507-a67f-4083-a6a1-a7389d66a99b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:42:59.588254 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d004507-a67f-4083-a6a1-a7389d66a99b service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:42:59.598001 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1434/2851] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:42:59 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:43:07.836429 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e5ebe-cc26-472c-83f0-7d4ca40d4349 None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:43:07.836944 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e5ebe-cc26-472c-83f0-7d4ca40d4349 None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:43:07.837061 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e5ebe-cc26-472c-83f0-7d4ca40d4349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:43:07.856513 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-0f4e5ebe-cc26-472c-83f0-7d4ca40d4349 None None] MFA Rules not processed for user `c7c4f8f2298d4014b2a08aaa391751f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:43:07.857730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-0f4e5ebe-cc26-472c-83f0-7d4ca40d4349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:43:07.914927 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1418/2852] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:43:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:43:07.950460 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d004507-a67f-4083-a6a1-a7389d66a99b service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:43:07.951920 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-2d004507-a67f-4083-a6a1-a7389d66a99b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:43:08.015876 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d004507-a67f-4083-a6a1-a7389d66a99b service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:43:08.020337 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41d998-3561-404a-b0db-d0a05dd64ce9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2483a351f5e943ad8c872a98a4717b0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:43:08.020996 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41d998-3561-404a-b0db-d0a05dd64ce9 service glance] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:43:08.021281 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41d998-3561-404a-b0db-d0a05dd64ce9 service glance] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:43:08.021693 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41d998-3561-404a-b0db-d0a05dd64ce9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:43:08.027605 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3a41d998-3561-404a-b0db-d0a05dd64ce9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:43:08.038656 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41d998-3561-404a-b0db-d0a05dd64ce9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:43:08.040096 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41d998-3561-404a-b0db-d0a05dd64ce9 service glance] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:43:08.051619 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1435/2853] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:43:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:43:08.572610 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e5ebe-cc26-472c-83f0-7d4ca40d4349 None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:43:08.584048 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e5ebe-cc26-472c-83f0-7d4ca40d4349 None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:43:08.588455 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3174f5-91f1-490c-acda-28809d2c1ede service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:43:08.589360 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3174f5-91f1-490c-acda-28809d2c1ede service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:43:08.589647 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3174f5-91f1-490c-acda-28809d2c1ede service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:43:08.589957 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3174f5-91f1-490c-acda-28809d2c1ede service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:43:08.591773 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-cb3174f5-91f1-490c-acda-28809d2c1ede service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:43:08.606296 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3174f5-91f1-490c-acda-28809d2c1ede service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:43:08.607982 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3174f5-91f1-490c-acda-28809d2c1ede service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:43:08.620780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1419/2854] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:43:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:43:18.046306 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=52 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:43:18.615103 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=31 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:44:06.370128 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41d998-3561-404a-b0db-d0a05dd64ce9 service glance] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:44:06.371612 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3a41d998-3561-404a-b0db-d0a05dd64ce9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:44:06.440966 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41d998-3561-404a-b0db-d0a05dd64ce9 service glance] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:44:06.445620 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544844c1-e57f-4f3f-83d0-49718c927098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:44:06.446498 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544844c1-e57f-4f3f-83d0-49718c927098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:06.446788 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544844c1-e57f-4f3f-83d0-49718c927098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:06.447078 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544844c1-e57f-4f3f-83d0-49718c927098 admin admin] PATH_INFO: `/v3/users/c7c4f8f2298d4014b2a08aaa391751f9` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:06.448390 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544844c1-e57f-4f3f-83d0-49718c927098 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7c4f8f2298d4014b2a08aaa391751f9)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:44:06.450341 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544844c1-e57f-4f3f-83d0-49718c927098 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:44:06.498222 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-544844c1-e57f-4f3f-83d0-49718c927098 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7c4f8f2298d4014b2a08aaa391751f9'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:44:06.509991 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-544844c1-e57f-4f3f-83d0-49718c927098 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7c4f8f2298d4014b2a08aaa391751f9'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:44:06.518362 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-544844c1-e57f-4f3f-83d0-49718c927098 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7c4f8f2298d4014b2a08aaa391751f9'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:44:06.519303 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-544844c1-e57f-4f3f-83d0-49718c927098 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7c4f8f2298d4014b2a08aaa391751f9'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:44:06.520135 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-544844c1-e57f-4f3f-83d0-49718c927098 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7c4f8f2298d4014b2a08aaa391751f9'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:44:06.533237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1436/2855] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:44:06 2022] DELETE /identity/v3/users/c7c4f8f2298d4014b2a08aaa391751f9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:44:06.711583 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3174f5-91f1-490c-acda-28809d2c1ede service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:44:06.725661 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3174f5-91f1-490c-acda-28809d2c1ede service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:44:06.793789 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd049a3-02ed-49e0-87e1-3b0ff2be1a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:44:06.794659 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd049a3-02ed-49e0-87e1-3b0ff2be1a84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:06.794951 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd049a3-02ed-49e0-87e1-3b0ff2be1a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:06.795234 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd049a3-02ed-49e0-87e1-3b0ff2be1a84 admin admin] PATH_INFO: `/v3/projects/d05c15fec3294719b7f8bc5690f355f9` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:06.796453 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd049a3-02ed-49e0-87e1-3b0ff2be1a84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d05c15fec3294719b7f8bc5690f355f9)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:44:06.798411 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd049a3-02ed-49e0-87e1-3b0ff2be1a84 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:44:06.845617 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2dd049a3-02ed-49e0-87e1-3b0ff2be1a84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd05c15fec3294719b7f8bc5690f355f9', 'request_id': 'req-2dd049a3-02ed-49e0-87e1-3b0ff2be1a84'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:44:06.853069 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2dd049a3-02ed-49e0-87e1-3b0ff2be1a84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd05c15fec3294719b7f8bc5690f355f9', 'request_id': 'req-2dd049a3-02ed-49e0-87e1-3b0ff2be1a84'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:44:06.860149 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2dd049a3-02ed-49e0-87e1-3b0ff2be1a84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd05c15fec3294719b7f8bc5690f355f9', 'request_id': 'req-2dd049a3-02ed-49e0-87e1-3b0ff2be1a84'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:44:06.870815 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2dd049a3-02ed-49e0-87e1-3b0ff2be1a84 admin admin] The token cache is being invalidate because project d05c15fec3294719b7f8bc5690f355f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:44:06.871100 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2dd049a3-02ed-49e0-87e1-3b0ff2be1a84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:44:06.871759 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-2dd049a3-02ed-49e0-87e1-3b0ff2be1a84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:44:06.873357 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1420/2856] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:44:06 2022] DELETE /identity/v3/projects/d05c15fec3294719b7f8bc5690f355f9 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:44:06.878398 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa9232-7d20-477d-94a7-fa4dcf227558 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:06.878726 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa9232-7d20-477d-94a7-fa4dcf227558 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:06.879014 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa9232-7d20-477d-94a7-fa4dcf227558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:06.898389 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-62fa9232-7d20-477d-94a7-fa4dcf227558 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:44:06.899607 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-62fa9232-7d20-477d-94a7-fa4dcf227558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:44:06.960382 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1437/2857] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:44:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:44:06.968642 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b5799-dbe3-4ea9-9377-e0914ef7ebfa None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:06.968931 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b5799-dbe3-4ea9-9377-e0914ef7ebfa None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:06.969215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b5799-dbe3-4ea9-9377-e0914ef7ebfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:06.989736 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-c42b5799-dbe3-4ea9-9377-e0914ef7ebfa None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:44:06.990938 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-c42b5799-dbe3-4ea9-9377-e0914ef7ebfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:44:06.998948 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1421/2858] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:44:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:44:07.003221 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa9232-7d20-477d-94a7-fa4dcf227558 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:44:07.013524 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa9232-7d20-477d-94a7-fa4dcf227558 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:44:07.018107 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8710940e-87b0-409f-8ce1-68b3bd2bf0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:44:07.018799 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8710940e-87b0-409f-8ce1-68b3bd2bf0be admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:07.019081 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8710940e-87b0-409f-8ce1-68b3bd2bf0be admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:07.019360 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8710940e-87b0-409f-8ce1-68b3bd2bf0be admin admin] PATH_INFO: `/v3/domains` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:07.020089 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8710940e-87b0-409f-8ce1-68b3bd2bf0be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80215) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 12:44:07.020424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8710940e-87b0-409f-8ce1-68b3bd2bf0be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:44:07.022033 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8710940e-87b0-409f-8ce1-68b3bd2bf0be admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:44:07.035194 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1438/2859] 10.222.0.16 () {58 vars in 1152 bytes} [Mon May 23 12:44:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 12:44:07.040046 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b5799-dbe3-4ea9-9377-e0914ef7ebfa None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:44:07.050658 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b5799-dbe3-4ea9-9377-e0914ef7ebfa None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:44:07.055332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0143f8-a3ec-4793-9a0f-2f1546263d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:44:07.055989 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0143f8-a3ec-4793-9a0f-2f1546263d30 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:07.056302 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0143f8-a3ec-4793-9a0f-2f1546263d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:07.056581 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0143f8-a3ec-4793-9a0f-2f1546263d30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:07.057767 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0143f8-a3ec-4793-9a0f-2f1546263d30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:44:07.059965 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0143f8-a3ec-4793-9a0f-2f1546263d30 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:44:07.087998 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1422/2860] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:44:07 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:44:07.092674 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8710940e-87b0-409f-8ce1-68b3bd2bf0be admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:44:07.102715 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8710940e-87b0-409f-8ce1-68b3bd2bf0be admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:44:07.106835 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:44:07.162384 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924de58-fd99-4c3c-92a0-5ed5c9c4bcc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:44:07.163059 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924de58-fd99-4c3c-92a0-5ed5c9c4bcc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:07.163351 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924de58-fd99-4c3c-92a0-5ed5c9c4bcc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:07.163630 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924de58-fd99-4c3c-92a0-5ed5c9c4bcc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:07.164376 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3924de58-fd99-4c3c-92a0-5ed5c9c4bcc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:44:07.166300 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3924de58-fd99-4c3c-92a0-5ed5c9c4bcc4 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:44:07.188148 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1439/2861] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:44:07 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:44:07.192649 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0143f8-a3ec-4793-9a0f-2f1546263d30 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:44:07.202952 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0143f8-a3ec-4793-9a0f-2f1546263d30 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:44:07.207759 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d34a1-1e2e-4106-8730-a803f3775d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:44:07.208455 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d34a1-1e2e-4106-8730-a803f3775d45 admin admin] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:07.208742 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d34a1-1e2e-4106-8730-a803f3775d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:07.209021 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d34a1-1e2e-4106-8730-a803f3775d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:07.209750 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420d34a1-1e2e-4106-8730-a803f3775d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:44:07.211354 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420d34a1-1e2e-4106-8730-a803f3775d45 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:44:07.222111 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1423/2862] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:44:07.226525 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924de58-fd99-4c3c-92a0-5ed5c9c4bcc4 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:44:07.237483 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924de58-fd99-4c3c-92a0-5ed5c9c4bcc4 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:44:07.242090 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f023c7b7-36c5-4282-910b-c1898993e2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:44:07.242746 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023c7b7-36c5-4282-910b-c1898993e2ee admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:07.243031 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023c7b7-36c5-4282-910b-c1898993e2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:07.243308 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023c7b7-36c5-4282-910b-c1898993e2ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:07.244059 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f023c7b7-36c5-4282-910b-c1898993e2ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:44:07.246448 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f023c7b7-36c5-4282-910b-c1898993e2ee admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:44:07.256478 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1440/2863] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:44:07.260871 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d34a1-1e2e-4106-8730-a803f3775d45 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:44:07.270950 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d34a1-1e2e-4106-8730-a803f3775d45 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:44:07.275660 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8558ea6-c15a-4974-bd98-0c62a41eb62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:44:07.276690 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8558ea6-c15a-4974-bd98-0c62a41eb62f admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:07.277097 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8558ea6-c15a-4974-bd98-0c62a41eb62f admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:07.277393 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8558ea6-c15a-4974-bd98-0c62a41eb62f admin admin] PATH_INFO: `/v3/projects/cf51a0c52de248fdb85a7cee23739b4e/users/fd9f4184defc406e8fd753eac219341a/roles/504a45d998f0464893c90fa5429275a8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:07.292567 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8558ea6-c15a-4974-bd98-0c62a41eb62f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf51a0c52de248fdb85a7cee23739b4e, user_id=fd9f4184defc406e8fd753eac219341a, role_id=504a45d998f0464893c90fa5429275a8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:44:07.295261 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8558ea6-c15a-4974-bd98-0c62a41eb62f admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:44:07.309405 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1424/2864] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:44:07 2022] PUT /identity/v3/projects/cf51a0c52de248fdb85a7cee23739b4e/users/fd9f4184defc406e8fd753eac219341a/roles/504a45d998f0464893c90fa5429275a8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:44:07.316212 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b228a-0bd4-452b-935b-8cc4cebb36f4 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:07.316615 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b228a-0bd4-452b-935b-8cc4cebb36f4 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:07.316992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b228a-0bd4-452b-935b-8cc4cebb36f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:07.337297 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-1d1b228a-0bd4-452b-935b-8cc4cebb36f4 None None] MFA Rules not processed for user `89b57772ff384ce6af49426c38787139`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:44:07.338948 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-1d1b228a-0bd4-452b-935b-8cc4cebb36f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:44:07.407169 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1441/2865] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:44:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:44:07.417293 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8558ea6-c15a-4974-bd98-0c62a41eb62f admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:44:07.419264 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-a8558ea6-c15a-4974-bd98-0c62a41eb62f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:44:07.450370 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:44:07.481574 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8558ea6-c15a-4974-bd98-0c62a41eb62f admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:44:07.485251 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831cb00-90a0-47d3-a745-29654926ec1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:44:07.485897 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831cb00-90a0-47d3-a745-29654926ec1b service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:07.486181 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831cb00-90a0-47d3-a745-29654926ec1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:07.486458 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831cb00-90a0-47d3-a745-29654926ec1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:07.496157 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d831cb00-90a0-47d3-a745-29654926ec1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:44:07.497625 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d831cb00-90a0-47d3-a745-29654926ec1b service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:44:07.507048 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1425/2866] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:44:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:44:09.832665 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b228a-0bd4-452b-935b-8cc4cebb36f4 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:44:09.851266 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b228a-0bd4-452b-935b-8cc4cebb36f4 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:44:09.856678 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07adc1f7-7d22-45b4-96a7-0692fbe9bd23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:44:09.857535 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07adc1f7-7d22-45b4-96a7-0692fbe9bd23 service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:09.857976 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07adc1f7-7d22-45b4-96a7-0692fbe9bd23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:09.858418 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07adc1f7-7d22-45b4-96a7-0692fbe9bd23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:09.860587 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-07adc1f7-7d22-45b4-96a7-0692fbe9bd23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:44:09.874589 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07adc1f7-7d22-45b4-96a7-0692fbe9bd23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:44:09.879834 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07adc1f7-7d22-45b4-96a7-0692fbe9bd23 service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:44:09.891963 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1442/2867] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:44:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:44:12.329674 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7484e4-6307-4215-9c6b-bfffc9d909dc None None] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:12.330270 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7484e4-6307-4215-9c6b-bfffc9d909dc None None] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:12.331016 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7484e4-6307-4215-9c6b-bfffc9d909dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:12.366643 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.auth.core [None req-ae7484e4-6307-4215-9c6b-bfffc9d909dc None None] MFA Rules not processed for user `fd9f4184defc406e8fd753eac219341a`. Rule list: `[]` (Enabled: `True`). {{(pid=80216) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:44:12.367876 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-ae7484e4-6307-4215-9c6b-bfffc9d909dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:44:12.447380 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1426/2868] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:44:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:44:12.451987 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07adc1f7-7d22-45b4-96a7-0692fbe9bd23 service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:44:12.464236 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07adc1f7-7d22-45b4-96a7-0692fbe9bd23 service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:44:12.471318 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb33be8-6ad0-47bb-8418-00091352cc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:44:12.472467 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb33be8-6ad0-47bb-8418-00091352cc11 admin admin] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:12.472966 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb33be8-6ad0-47bb-8418-00091352cc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:12.473437 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb33be8-6ad0-47bb-8418-00091352cc11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:12.474711 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb33be8-6ad0-47bb-8418-00091352cc11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:44:12.477613 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb33be8-6ad0-47bb-8418-00091352cc11 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:44:12.512159 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1443/2869] 10.222.0.16 () {60 vars in 1152 bytes} [Mon May 23 12:44:12 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 12:44:12.516706 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7484e4-6307-4215-9c6b-bfffc9d909dc None None] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:44:12.530332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7484e4-6307-4215-9c6b-bfffc9d909dc None None] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:44:12.595776 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bdc6d-4b99-4143-8a95-5ed1924becc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:44:12.596662 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bdc6d-4b99-4143-8a95-5ed1924becc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:12.596832 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bdc6d-4b99-4143-8a95-5ed1924becc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:12.597069 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bdc6d-4b99-4143-8a95-5ed1924becc3 admin admin] PATH_INFO: `/v3/users` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:12.597780 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58bdc6d-4b99-4143-8a95-5ed1924becc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:44:12.599786 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58bdc6d-4b99-4143-8a95-5ed1924becc3 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:44:12.628104 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1427/2870] 10.222.0.16 () {60 vars in 1143 bytes} [Mon May 23 12:44:12 2022] POST /identity/v3/users => generated 298 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 12:44:12.629614 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb33be8-6ad0-47bb-8418-00091352cc11 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:44:12.641308 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb33be8-6ad0-47bb-8418-00091352cc11 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:44:12.648197 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6316e6-c13a-489d-9130-9f7c123a6787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:44:12.649320 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6316e6-c13a-489d-9130-9f7c123a6787 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:12.649815 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6316e6-c13a-489d-9130-9f7c123a6787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:12.650294 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6316e6-c13a-489d-9130-9f7c123a6787 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:12.651641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6316e6-c13a-489d-9130-9f7c123a6787 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:44:12.654631 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6316e6-c13a-489d-9130-9f7c123a6787 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:44:12.684806 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1444/2871] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 12:44:12.685824 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bdc6d-4b99-4143-8a95-5ed1924becc3 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:44:12.704373 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bdc6d-4b99-4143-8a95-5ed1924becc3 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:44:12.714110 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d5853-0637-4513-9c32-049546f8c545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:44:12.715611 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d5853-0637-4513-9c32-049546f8c545 admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:12.716154 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d5853-0637-4513-9c32-049546f8c545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:12.716602 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d5853-0637-4513-9c32-049546f8c545 admin admin] PATH_INFO: `/v3/projects/f0bd4c0da82f4ea5abc3097525de240b/users/9046b7ccc83f44bcaeff3106ce28e311/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:12.740242 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614d5853-0637-4513-9c32-049546f8c545 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0bd4c0da82f4ea5abc3097525de240b, user_id=9046b7ccc83f44bcaeff3106ce28e311, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:44:12.743291 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614d5853-0637-4513-9c32-049546f8c545 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:44:12.763138 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1428/2872] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:44:12 2022] PUT /identity/v3/projects/f0bd4c0da82f4ea5abc3097525de240b/users/9046b7ccc83f44bcaeff3106ce28e311/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:44:12.767097 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6316e6-c13a-489d-9130-9f7c123a6787 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:44:12.778199 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6316e6-c13a-489d-9130-9f7c123a6787 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:44:12.824633 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:44:12.856279 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab277fe-242f-4672-8fd6-75784454bd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:44:12.857463 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab277fe-242f-4672-8fd6-75784454bd91 admin admin] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:12.857815 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab277fe-242f-4672-8fd6-75784454bd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:12.858151 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab277fe-242f-4672-8fd6-75784454bd91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:12.859271 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab277fe-242f-4672-8fd6-75784454bd91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:44:12.861119 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab277fe-242f-4672-8fd6-75784454bd91 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:44:12.874137 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1445/2873] 10.222.0.16 () {58 vars in 1121 bytes} [Mon May 23 12:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 12:44:12.878327 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d5853-0637-4513-9c32-049546f8c545 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:44:12.890490 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d5853-0637-4513-9c32-049546f8c545 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:44:12.895019 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16600b-e2e9-44c7-be65-f5dbab6b8c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:44:12.895926 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16600b-e2e9-44c7-be65-f5dbab6b8c9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:12.896239 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16600b-e2e9-44c7-be65-f5dbab6b8c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:12.896515 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16600b-e2e9-44c7-be65-f5dbab6b8c9d admin admin] PATH_INFO: `/v3/projects/f0bd4c0da82f4ea5abc3097525de240b/users/9046b7ccc83f44bcaeff3106ce28e311/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:12.898506 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a16600b-e2e9-44c7-be65-f5dbab6b8c9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0bd4c0da82f4ea5abc3097525de240b, user_id=9046b7ccc83f44bcaeff3106ce28e311, role_id=4e085d2ed7164b56aa2e4e4d85d0c7f8)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:44:12.901842 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a16600b-e2e9-44c7-be65-f5dbab6b8c9d admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:44:12.915363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1429/2874] 10.222.0.16 () {60 vars in 1482 bytes} [Mon May 23 12:44:12 2022] PUT /identity/v3/projects/f0bd4c0da82f4ea5abc3097525de240b/users/9046b7ccc83f44bcaeff3106ce28e311/roles/4e085d2ed7164b56aa2e4e4d85d0c7f8 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:44:18.251364 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231c56e-fab2-4ee1-87d9-8ba62420b319 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:18.252483 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231c56e-fab2-4ee1-87d9-8ba62420b319 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:18.252483 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231c56e-fab2-4ee1-87d9-8ba62420b319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:18.297922 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-4231c56e-fab2-4ee1-87d9-8ba62420b319 None None] MFA Rules not processed for user `9046b7ccc83f44bcaeff3106ce28e311`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:44:18.299237 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-4231c56e-fab2-4ee1-87d9-8ba62420b319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:44:18.396823 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1446/2875] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:44:18 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:44:18.414239 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16600b-e2e9-44c7-be65-f5dbab6b8c9d admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:44:18.416026 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-1a16600b-e2e9-44c7-be65-f5dbab6b8c9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:44:18.439509 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:44:18.499845 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16600b-e2e9-44c7-be65-f5dbab6b8c9d admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:44:18.503772 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b7952-e416-4eda-9468-1a56a6dad270 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2483a351f5e943ad8c872a98a4717b0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:44:18.504545 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b7952-e416-4eda-9468-1a56a6dad270 service glance] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:18.504831 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b7952-e416-4eda-9468-1a56a6dad270 service glance] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:18.505112 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b7952-e416-4eda-9468-1a56a6dad270 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:18.570998 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b7952-e416-4eda-9468-1a56a6dad270 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:44:18.573113 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b7952-e416-4eda-9468-1a56a6dad270 service glance] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:44:18.584216 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1430/2876] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:44:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:44:18.699401 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea00b68-2ea3-4590-aaeb-b6996758c82b None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:18.699974 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea00b68-2ea3-4590-aaeb-b6996758c82b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:18.700406 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea00b68-2ea3-4590-aaeb-b6996758c82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:18.732606 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-bea00b68-2ea3-4590-aaeb-b6996758c82b None None] MFA Rules not processed for user `733f8296ec3d4795a1140c1e1cb6eb95`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:44:18.733938 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-bea00b68-2ea3-4590-aaeb-b6996758c82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:44:18.809972 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1447/2877] 10.222.0.16 () {58 vars in 1009 bytes} [Mon May 23 12:44:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:44:18.870791 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b7952-e416-4eda-9468-1a56a6dad270 service glance] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:44:18.872385 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-6b6b7952-e416-4eda-9468-1a56a6dad270 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:44:18.952761 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b7952-e416-4eda-9468-1a56a6dad270 service glance] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:44:18.958630 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4527267-ac38-464f-ad2d-bcbe740de3a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71fb9a79ac494635804b93d4a0048205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:44:18.959312 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4527267-ac38-464f-ad2d-bcbe740de3a7 service swift] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:18.959620 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4527267-ac38-464f-ad2d-bcbe740de3a7 service swift] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:18.959922 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4527267-ac38-464f-ad2d-bcbe740de3a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:18.970708 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4527267-ac38-464f-ad2d-bcbe740de3a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:44:18.972278 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4527267-ac38-464f-ad2d-bcbe740de3a7 service swift] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:44:18.982417 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1431/2878] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 12:44:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 12:44:27.523246 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442335d8-373d-4cfd-92b1-fb2e1c0cb9c7 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:27.523796 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442335d8-373d-4cfd-92b1-fb2e1c0cb9c7 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:27.523796 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442335d8-373d-4cfd-92b1-fb2e1c0cb9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:27.542366 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-442335d8-373d-4cfd-92b1-fb2e1c0cb9c7 None None] MFA Rules not processed for user `fd9f4184defc406e8fd753eac219341a`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:44:27.543583 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-442335d8-373d-4cfd-92b1-fb2e1c0cb9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:44:27.550347 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1448/2879] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:44:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:44:28.143783 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4527267-ac38-464f-ad2d-bcbe740de3a7 service swift] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:44:28.145321 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-e4527267-ac38-464f-ad2d-bcbe740de3a7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:44:28.206654 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=100 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:44:28.226322 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4527267-ac38-464f-ad2d-bcbe740de3a7 service swift] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:44:28.231509 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f012787-cc8e-42c7-a625-84ed30a520de service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc0f137e0d24dd2a460689bc17a1b98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:44:28.232239 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f012787-cc8e-42c7-a625-84ed30a520de service placement] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:28.232527 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f012787-cc8e-42c7-a625-84ed30a520de service placement] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:28.232829 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f012787-cc8e-42c7-a625-84ed30a520de service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:28.234586 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-8f012787-cc8e-42c7-a625-84ed30a520de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:44:28.246283 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f012787-cc8e-42c7-a625-84ed30a520de service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:44:28.247863 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f012787-cc8e-42c7-a625-84ed30a520de service placement] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:44:28.257930 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1432/2880] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:44:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:44:29.375895 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74391b3e-a1df-43e3-a7a7-7436fc4cdbf9 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:29.376441 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74391b3e-a1df-43e3-a7a7-7436fc4cdbf9 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:29.376540 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74391b3e-a1df-43e3-a7a7-7436fc4cdbf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:29.395420 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-74391b3e-a1df-43e3-a7a7-7436fc4cdbf9 None None] MFA Rules not processed for user `733f8296ec3d4795a1140c1e1cb6eb95`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:44:29.396655 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-74391b3e-a1df-43e3-a7a7-7436fc4cdbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:44:29.404357 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1449/2881] 10.222.0.16 () {58 vars in 1009 bytes} [Mon May 23 12:44:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:44:29.416385 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f012787-cc8e-42c7-a625-84ed30a520de service placement] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:44:29.437079 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f012787-cc8e-42c7-a625-84ed30a520de service placement] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:44:29.442275 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406c852-91fd-44ea-b832-6b88b213c35a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71fb9a79ac494635804b93d4a0048205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:44:29.442996 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406c852-91fd-44ea-b832-6b88b213c35a service swift] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:29.443353 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406c852-91fd-44ea-b832-6b88b213c35a service swift] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:29.443701 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a406c852-91fd-44ea-b832-6b88b213c35a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:29.452915 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406c852-91fd-44ea-b832-6b88b213c35a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:44:29.454562 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a406c852-91fd-44ea-b832-6b88b213c35a service swift] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:44:29.464694 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1433/2882] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 12:44:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 12:44:37.865167 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74391b3e-a1df-43e3-a7a7-7436fc4cdbf9 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:44:37.866943 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-74391b3e-a1df-43e3-a7a7-7436fc4cdbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:44:37.931859 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=100 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:44:37.958027 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74391b3e-a1df-43e3-a7a7-7436fc4cdbf9 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:44:37.964705 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676210c6-ef2f-4577-91cf-acc8a0083fd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:44:37.965422 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676210c6-ef2f-4577-91cf-acc8a0083fd3 service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:37.965714 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676210c6-ef2f-4577-91cf-acc8a0083fd3 service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:37.966010 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676210c6-ef2f-4577-91cf-acc8a0083fd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:37.968105 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-676210c6-ef2f-4577-91cf-acc8a0083fd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:44:37.985896 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676210c6-ef2f-4577-91cf-acc8a0083fd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:44:37.987832 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676210c6-ef2f-4577-91cf-acc8a0083fd3 service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:44:38.001338 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1450/2883] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:44:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:44:39.460301 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=13 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:44:44.382727 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406c852-91fd-44ea-b832-6b88b213c35a service swift] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:44:44.411459 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a406c852-91fd-44ea-b832-6b88b213c35a service swift] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:44:44.417041 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f546458-fb12-426a-9eab-fdb90b6f227d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:44:44.417698 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f546458-fb12-426a-9eab-fdb90b6f227d service nova] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:44.417987 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f546458-fb12-426a-9eab-fdb90b6f227d service nova] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:44.418271 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f546458-fb12-426a-9eab-fdb90b6f227d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:44.420085 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-9f546458-fb12-426a-9eab-fdb90b6f227d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:44:44.448834 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f546458-fb12-426a-9eab-fdb90b6f227d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:44:44.450655 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f546458-fb12-426a-9eab-fdb90b6f227d service nova] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:44:44.462677 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1434/2884] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:44:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 12:44:47.996660 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=10 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:44:54.457644 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=16 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:44:56.067706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676210c6-ef2f-4577-91cf-acc8a0083fd3 service nova] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:44:56.152150 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676210c6-ef2f-4577-91cf-acc8a0083fd3 service nova] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:44:56.156578 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c82837-fc42-4c19-a841-5769c9a6057f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:44:56.157226 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c82837-fc42-4c19-a841-5769c9a6057f service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:56.157509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c82837-fc42-4c19-a841-5769c9a6057f service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:56.157785 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c82837-fc42-4c19-a841-5769c9a6057f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:56.190367 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c82837-fc42-4c19-a841-5769c9a6057f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:44:56.191793 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c82837-fc42-4c19-a841-5769c9a6057f service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:44:56.201624 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1451/2885] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:44:56 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:44:59.561581 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f546458-fb12-426a-9eab-fdb90b6f227d service nova] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:44:59.621275 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f546458-fb12-426a-9eab-fdb90b6f227d service nova] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:44:59.628215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d960c2-6f41-4303-8757-22eddea12b42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:44:59.628885 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d960c2-6f41-4303-8757-22eddea12b42 service nova] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:44:59.629177 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d960c2-6f41-4303-8757-22eddea12b42 service nova] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:44:59.629460 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d960c2-6f41-4303-8757-22eddea12b42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:44:59.631332 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-75d960c2-6f41-4303-8757-22eddea12b42 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:44:59.647839 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d960c2-6f41-4303-8757-22eddea12b42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:44:59.650298 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d960c2-6f41-4303-8757-22eddea12b42 service nova] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:44:59.666939 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1435/2886] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:44:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:45:06.196918 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=45 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:45:09.534331 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55170b7-155a-45b8-b513-82e1e11e3d41 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:45:09.534951 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55170b7-155a-45b8-b513-82e1e11e3d41 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:45:09.534951 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55170b7-155a-45b8-b513-82e1e11e3d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:45:09.551995 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-e55170b7-155a-45b8-b513-82e1e11e3d41 None None] MFA Rules not processed for user `024cf85a89d445ff854fede06d0f9244`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:45:09.553210 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e55170b7-155a-45b8-b513-82e1e11e3d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:45:09.611404 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1452/2887] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:45:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:45:09.658711 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=14 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:45:19.604521 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=25 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:46:20.126032 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d960c2-6f41-4303-8757-22eddea12b42 service nova] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:46:20.127433 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-75d960c2-6f41-4303-8757-22eddea12b42 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:46:20.164957 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d960c2-6f41-4303-8757-22eddea12b42 service nova] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:46:20.170752 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31384500-828e-485f-8e22-96ded6e4ba0f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc0f137e0d24dd2a460689bc17a1b98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:46:20.171410 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31384500-828e-485f-8e22-96ded6e4ba0f service placement] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:46:20.171700 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31384500-828e-485f-8e22-96ded6e4ba0f service placement] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:46:20.171981 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31384500-828e-485f-8e22-96ded6e4ba0f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:46:20.173796 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-31384500-828e-485f-8e22-96ded6e4ba0f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:46:20.186556 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31384500-828e-485f-8e22-96ded6e4ba0f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:46:20.188195 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31384500-828e-485f-8e22-96ded6e4ba0f service placement] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:46:20.204745 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1436/2888] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:46:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:46:21.183450 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa169e-a48a-4c41-bb31-c7286dd6834b None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:46:21.183991 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa169e-a48a-4c41-bb31-c7286dd6834b None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:46:21.184264 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa169e-a48a-4c41-bb31-c7286dd6834b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:46:21.243250 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-64fa169e-a48a-4c41-bb31-c7286dd6834b None None] MFA Rules not processed for user `733f8296ec3d4795a1140c1e1cb6eb95`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:46:21.244518 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-64fa169e-a48a-4c41-bb31-c7286dd6834b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:46:21.269563 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1453/2889] 10.222.0.16 () {58 vars in 967 bytes} [Mon May 23 12:46:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:46:21.309993 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31384500-828e-485f-8e22-96ded6e4ba0f service placement] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:46:21.331814 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31384500-828e-485f-8e22-96ded6e4ba0f service placement] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:46:21.336888 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9ed63-6551-4699-aa94-dadd39e372e8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71fb9a79ac494635804b93d4a0048205', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:46:21.337538 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9ed63-6551-4699-aa94-dadd39e372e8 service swift] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:46:21.337827 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9ed63-6551-4699-aa94-dadd39e372e8 service swift] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:46:21.338106 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9ed63-6551-4699-aa94-dadd39e372e8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:46:21.347281 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe9ed63-6551-4699-aa94-dadd39e372e8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:46:21.348903 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe9ed63-6551-4699-aa94-dadd39e372e8 service swift] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:46:21.357293 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1437/2890] 10.222.0.16 () {60 vars in 1384 bytes} [Mon May 23 12:46:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 23 12:46:24.835972 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa169e-a48a-4c41-bb31-c7286dd6834b None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:46:24.849799 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa169e-a48a-4c41-bb31-c7286dd6834b None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:46:24.856533 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96745fa-526f-4797-a061-6c43b9e8b304 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc0f137e0d24dd2a460689bc17a1b98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:46:24.857196 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96745fa-526f-4797-a061-6c43b9e8b304 service placement] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:46:24.858578 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96745fa-526f-4797-a061-6c43b9e8b304 service placement] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:46:24.859324 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96745fa-526f-4797-a061-6c43b9e8b304 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:46:24.861609 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e96745fa-526f-4797-a061-6c43b9e8b304 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:46:24.886841 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96745fa-526f-4797-a061-6c43b9e8b304 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:46:24.889477 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96745fa-526f-4797-a061-6c43b9e8b304 service placement] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:46:24.906057 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1454/2891] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:46:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:46:28.131387 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9ed63-6551-4699-aa94-dadd39e372e8 service swift] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:46:28.143903 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9ed63-6551-4699-aa94-dadd39e372e8 service swift] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:46:28.149122 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944436b-c889-4273-a9b1-13a9de6c088a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc0f137e0d24dd2a460689bc17a1b98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:46:28.149881 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944436b-c889-4273-a9b1-13a9de6c088a service placement] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:46:28.150169 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944436b-c889-4273-a9b1-13a9de6c088a service placement] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:46:28.150508 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944436b-c889-4273-a9b1-13a9de6c088a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:46:28.153236 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-b944436b-c889-4273-a9b1-13a9de6c088a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:46:28.167152 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b944436b-c889-4273-a9b1-13a9de6c088a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:46:28.167968 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b944436b-c889-4273-a9b1-13a9de6c088a service placement] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:46:28.182784 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1438/2892] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:46:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:46:28.189888 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96745fa-526f-4797-a061-6c43b9e8b304 service placement] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:46:28.284929 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96745fa-526f-4797-a061-6c43b9e8b304 service placement] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:46:28.289538 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e169f11-d830-4aa2-a6d7-b455a144b483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:46:28.290414 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e169f11-d830-4aa2-a6d7-b455a144b483 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:46:28.290734 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e169f11-d830-4aa2-a6d7-b455a144b483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:46:28.291019 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e169f11-d830-4aa2-a6d7-b455a144b483 admin admin] PATH_INFO: `/v3/users/fd9f4184defc406e8fd753eac219341a` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:46:28.292227 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e169f11-d830-4aa2-a6d7-b455a144b483 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd9f4184defc406e8fd753eac219341a)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:46:28.294769 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e169f11-d830-4aa2-a6d7-b455a144b483 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:46:28.339072 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3e169f11-d830-4aa2-a6d7-b455a144b483 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd9f4184defc406e8fd753eac219341a'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:46:28.351268 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3e169f11-d830-4aa2-a6d7-b455a144b483 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd9f4184defc406e8fd753eac219341a'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:46:28.359174 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3e169f11-d830-4aa2-a6d7-b455a144b483 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd9f4184defc406e8fd753eac219341a'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:46:28.359783 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3e169f11-d830-4aa2-a6d7-b455a144b483 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd9f4184defc406e8fd753eac219341a'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:46:28.360424 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-3e169f11-d830-4aa2-a6d7-b455a144b483 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd9f4184defc406e8fd753eac219341a'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:46:28.364932 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:46:28.374481 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1455/2893] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:46:28 2022] DELETE /identity/v3/users/fd9f4184defc406e8fd753eac219341a => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:46:28.545764 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944436b-c889-4273-a9b1-13a9de6c088a service placement] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:46:28.559279 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944436b-c889-4273-a9b1-13a9de6c088a service placement] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:46:28.620702 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a03f24-607a-476a-a292-84f0d03ddc05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:46:28.621692 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a03f24-607a-476a-a292-84f0d03ddc05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:46:28.621987 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a03f24-607a-476a-a292-84f0d03ddc05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:46:28.622275 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a03f24-607a-476a-a292-84f0d03ddc05 admin admin] PATH_INFO: `/v3/projects/cf51a0c52de248fdb85a7cee23739b4e` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:46:28.623460 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a03f24-607a-476a-a292-84f0d03ddc05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf51a0c52de248fdb85a7cee23739b4e)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:46:28.625493 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a03f24-607a-476a-a292-84f0d03ddc05 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:46:28.673106 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-37a03f24-607a-476a-a292-84f0d03ddc05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf51a0c52de248fdb85a7cee23739b4e', 'request_id': 'req-37a03f24-607a-476a-a292-84f0d03ddc05'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:46:28.683472 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-37a03f24-607a-476a-a292-84f0d03ddc05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf51a0c52de248fdb85a7cee23739b4e', 'request_id': 'req-37a03f24-607a-476a-a292-84f0d03ddc05'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:46:28.691215 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-37a03f24-607a-476a-a292-84f0d03ddc05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf51a0c52de248fdb85a7cee23739b4e', 'request_id': 'req-37a03f24-607a-476a-a292-84f0d03ddc05'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:46:28.716400 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-37a03f24-607a-476a-a292-84f0d03ddc05 admin admin] The token cache is being invalidate because project cf51a0c52de248fdb85a7cee23739b4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:46:28.716695 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-37a03f24-607a-476a-a292-84f0d03ddc05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:46:28.717363 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-37a03f24-607a-476a-a292-84f0d03ddc05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:46:28.718934 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1439/2894] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:46:28 2022] DELETE /identity/v3/projects/cf51a0c52de248fdb85a7cee23739b4e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:46:28.723227 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e169f11-d830-4aa2-a6d7-b455a144b483 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:46:28.724683 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-3e169f11-d830-4aa2-a6d7-b455a144b483 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:46:28.739825 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e169f11-d830-4aa2-a6d7-b455a144b483 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:46:28.813251 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2269fc9-b9c1-4aaa-bbe0-cd50fece5186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:46:28.814156 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2269fc9-b9c1-4aaa-bbe0-cd50fece5186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:46:28.814448 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2269fc9-b9c1-4aaa-bbe0-cd50fece5186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:46:28.814732 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2269fc9-b9c1-4aaa-bbe0-cd50fece5186 admin admin] PATH_INFO: `/v3/users/9046b7ccc83f44bcaeff3106ce28e311` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:46:28.815952 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2269fc9-b9c1-4aaa-bbe0-cd50fece5186 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9046b7ccc83f44bcaeff3106ce28e311)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:46:28.817940 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2269fc9-b9c1-4aaa-bbe0-cd50fece5186 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:46:28.877645 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d2269fc9-b9c1-4aaa-bbe0-cd50fece5186 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9046b7ccc83f44bcaeff3106ce28e311'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:46:28.889992 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d2269fc9-b9c1-4aaa-bbe0-cd50fece5186 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9046b7ccc83f44bcaeff3106ce28e311'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:46:28.897560 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d2269fc9-b9c1-4aaa-bbe0-cd50fece5186 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9046b7ccc83f44bcaeff3106ce28e311'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:46:28.898182 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d2269fc9-b9c1-4aaa-bbe0-cd50fece5186 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9046b7ccc83f44bcaeff3106ce28e311'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:46:28.898791 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-d2269fc9-b9c1-4aaa-bbe0-cd50fece5186 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9046b7ccc83f44bcaeff3106ce28e311'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:46:28.913196 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1456/2895] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:46:28 2022] DELETE /identity/v3/users/9046b7ccc83f44bcaeff3106ce28e311 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:46:29.098686 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a03f24-607a-476a-a292-84f0d03ddc05 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:46:29.100443 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-37a03f24-607a-476a-a292-84f0d03ddc05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:46:29.115328 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a03f24-607a-476a-a292-84f0d03ddc05 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:46:29.124880 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:46:29.178923 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a19126-76bf-4c57-bb3f-b205638ad7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:46:29.179795 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a19126-76bf-4c57-bb3f-b205638ad7ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:46:29.180087 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a19126-76bf-4c57-bb3f-b205638ad7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:46:29.180404 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a19126-76bf-4c57-bb3f-b205638ad7ac admin admin] PATH_INFO: `/v3/projects/f0bd4c0da82f4ea5abc3097525de240b` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:46:29.181600 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a19126-76bf-4c57-bb3f-b205638ad7ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0bd4c0da82f4ea5abc3097525de240b)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:46:29.183554 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a19126-76bf-4c57-bb3f-b205638ad7ac admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:46:29.232027 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-a0a19126-76bf-4c57-bb3f-b205638ad7ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0bd4c0da82f4ea5abc3097525de240b', 'request_id': 'req-a0a19126-76bf-4c57-bb3f-b205638ad7ac'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:46:29.239445 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-a0a19126-76bf-4c57-bb3f-b205638ad7ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0bd4c0da82f4ea5abc3097525de240b', 'request_id': 'req-a0a19126-76bf-4c57-bb3f-b205638ad7ac'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:46:29.245925 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-a0a19126-76bf-4c57-bb3f-b205638ad7ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0bd4c0da82f4ea5abc3097525de240b', 'request_id': 'req-a0a19126-76bf-4c57-bb3f-b205638ad7ac'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:46:29.256267 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-a0a19126-76bf-4c57-bb3f-b205638ad7ac admin admin] The token cache is being invalidate because project f0bd4c0da82f4ea5abc3097525de240b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:46:29.256557 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-a0a19126-76bf-4c57-bb3f-b205638ad7ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:46:29.257201 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-a0a19126-76bf-4c57-bb3f-b205638ad7ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:46:29.258805 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1440/2896] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:46:29 2022] DELETE /identity/v3/projects/f0bd4c0da82f4ea5abc3097525de240b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:46:38.909407 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=56,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:46:39.253299 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:47:04.935141 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998daa4b-7927-4ee0-9eee-ca6113f72689 None None] REQUEST_METHOD: `POST` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:47:04.935706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998daa4b-7927-4ee0-9eee-ca6113f72689 None None] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:47:04.935706 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998daa4b-7927-4ee0-9eee-ca6113f72689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:47:04.953694 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.auth.core [None req-998daa4b-7927-4ee0-9eee-ca6113f72689 None None] MFA Rules not processed for user `024cf85a89d445ff854fede06d0f9244`. Rule list: `[]` (Enabled: `True`). {{(pid=80215) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 12:47:04.954875 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-998daa4b-7927-4ee0-9eee-ca6113f72689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:47:05.010396 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1457/2897] 10.222.0.16 () {58 vars in 957 bytes} [Mon May 23 12:47:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 12:47:05.332225 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a19126-76bf-4c57-bb3f-b205638ad7ac admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:47:05.333630 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-a0a19126-76bf-4c57-bb3f-b205638ad7ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:47:05.409147 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a19126-76bf-4c57-bb3f-b205638ad7ac admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:47:05.412876 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982861a-edc5-454c-a35c-255fd6468028 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:47:05.413533 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982861a-edc5-454c-a35c-255fd6468028 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:47:05.413825 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982861a-edc5-454c-a35c-255fd6468028 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:47:05.414111 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982861a-edc5-454c-a35c-255fd6468028 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:47:05.415925 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-7982861a-edc5-454c-a35c-255fd6468028 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:47:05.426766 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7982861a-edc5-454c-a35c-255fd6468028 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:47:05.428232 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7982861a-edc5-454c-a35c-255fd6468028 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:47:05.436887 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1441/2898] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:47:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:47:15.003650 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=25 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:47:15.432803 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=31 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:47:17.640790 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998daa4b-7927-4ee0-9eee-ca6113f72689 None None] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:47:17.642247 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-998daa4b-7927-4ee0-9eee-ca6113f72689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:47:17.741008 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998daa4b-7927-4ee0-9eee-ca6113f72689 None None] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:47:17.746543 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb8be4-6010-433f-80e6-6d8fff56c68a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b83535b443a84275a160cd84cb3ad5a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:47:17.747993 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb8be4-6010-433f-80e6-6d8fff56c68a service nova] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:47:17.748420 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb8be4-6010-433f-80e6-6d8fff56c68a service nova] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:47:17.748700 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb8be4-6010-433f-80e6-6d8fff56c68a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:47:17.750470 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-42cb8be4-6010-433f-80e6-6d8fff56c68a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:47:17.760761 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb8be4-6010-433f-80e6-6d8fff56c68a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:47:17.762340 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb8be4-6010-433f-80e6-6d8fff56c68a service nova] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:47:17.771509 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1458/2899] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:47:17 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:47:27.767536 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=43 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:48:51.913007 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982861a-edc5-454c-a35c-255fd6468028 service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:48:51.923730 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982861a-edc5-454c-a35c-255fd6468028 service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:48:51.927742 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c051140-9350-42fe-8bee-1339f5cc2d75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:48:51.928557 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c051140-9350-42fe-8bee-1339f5cc2d75 service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:48:51.928849 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c051140-9350-42fe-8bee-1339f5cc2d75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:48:51.929131 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c051140-9350-42fe-8bee-1339f5cc2d75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:48:51.931004 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-0c051140-9350-42fe-8bee-1339f5cc2d75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:48:51.947138 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c051140-9350-42fe-8bee-1339f5cc2d75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:48:51.948809 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c051140-9350-42fe-8bee-1339f5cc2d75 service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:48:51.962203 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1442/2900] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:48:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:48:53.156331 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb8be4-6010-433f-80e6-6d8fff56c68a service nova] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:48:53.166566 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb8be4-6010-433f-80e6-6d8fff56c68a service nova] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:48:53.170348 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fe6c7-f254-4f51-9465-50f4c423156f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:48:53.171008 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fe6c7-f254-4f51-9465-50f4c423156f service neutron] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:48:53.171298 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fe6c7-f254-4f51-9465-50f4c423156f service neutron] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:48:53.171577 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638fe6c7-f254-4f51-9465-50f4c423156f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:48:53.173413 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-638fe6c7-f254-4f51-9465-50f4c423156f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:48:53.187574 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638fe6c7-f254-4f51-9465-50f4c423156f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:48:53.189045 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638fe6c7-f254-4f51-9465-50f4c423156f service neutron] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:48:53.200914 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1459/2901] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:48:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:49:01.955595 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=6 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:49:03.195427 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=6 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:49:04.490328 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c051140-9350-42fe-8bee-1339f5cc2d75 service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:49:04.491862 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-0c051140-9350-42fe-8bee-1339f5cc2d75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:49:04.504655 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c051140-9350-42fe-8bee-1339f5cc2d75 service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:49:04.508342 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc8bf8e-4c42-4aab-a4f4-6eb24f383f2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6677be3ddf647e2b971bb3ba2c34a3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:49:04.508989 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8bf8e-4c42-4aab-a4f4-6eb24f383f2f service neutron] REQUEST_METHOD: `GET` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:49:04.509340 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8bf8e-4c42-4aab-a4f4-6eb24f383f2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:49:04.510409 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8bf8e-4c42-4aab-a4f4-6eb24f383f2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:49:04.512672 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-4dc8bf8e-4c42-4aab-a4f4-6eb24f383f2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:49:04.580231 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc8bf8e-4c42-4aab-a4f4-6eb24f383f2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:49:04.581626 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc8bf8e-4c42-4aab-a4f4-6eb24f383f2f service neutron] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:49:04.592288 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1443/2902] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:49:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:49:11.234260 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fe6c7-f254-4f51-9465-50f4c423156f service neutron] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:49:11.235863 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-638fe6c7-f254-4f51-9465-50f4c423156f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:49:11.250403 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638fe6c7-f254-4f51-9465-50f4c423156f service neutron] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:49:11.255068 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d50ef0-05d8-4dba-8f15-dd2842e4d1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:49:11.255948 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d50ef0-05d8-4dba-8f15-dd2842e4d1e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:49:11.256267 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d50ef0-05d8-4dba-8f15-dd2842e4d1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:49:11.256732 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d50ef0-05d8-4dba-8f15-dd2842e4d1e2 admin admin] PATH_INFO: `/v3/users/024cf85a89d445ff854fede06d0f9244` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:49:11.258278 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d50ef0-05d8-4dba-8f15-dd2842e4d1e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=024cf85a89d445ff854fede06d0f9244)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:49:11.262108 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d50ef0-05d8-4dba-8f15-dd2842e4d1e2 admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:49:11.318202 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e8d50ef0-05d8-4dba-8f15-dd2842e4d1e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '024cf85a89d445ff854fede06d0f9244'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:49:11.330281 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e8d50ef0-05d8-4dba-8f15-dd2842e4d1e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '024cf85a89d445ff854fede06d0f9244'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:49:11.338926 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e8d50ef0-05d8-4dba-8f15-dd2842e4d1e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '024cf85a89d445ff854fede06d0f9244'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:49:11.339553 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e8d50ef0-05d8-4dba-8f15-dd2842e4d1e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '024cf85a89d445ff854fede06d0f9244'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:49:11.340206 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-e8d50ef0-05d8-4dba-8f15-dd2842e4d1e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '024cf85a89d445ff854fede06d0f9244'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:49:11.354114 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1460/2903] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:49:11 2022] DELETE /identity/v3/users/024cf85a89d445ff854fede06d0f9244 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 12:49:11.536378 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc8bf8e-4c42-4aab-a4f4-6eb24f383f2f service neutron] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:49:11.552076 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc8bf8e-4c42-4aab-a4f4-6eb24f383f2f service neutron] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:49:11.615367 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9766b-b421-4351-b271-be2d6bb22a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:49:11.616143 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9766b-b421-4351-b271-be2d6bb22a25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:49:11.616461 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9766b-b421-4351-b271-be2d6bb22a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:49:11.616743 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9766b-b421-4351-b271-be2d6bb22a25 admin admin] PATH_INFO: `/v3/projects/ac5975413f5548c3881ec3c868295836` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:49:11.617961 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b9766b-b421-4351-b271-be2d6bb22a25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac5975413f5548c3881ec3c868295836)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:49:11.620381 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b9766b-b421-4351-b271-be2d6bb22a25 admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:49:11.674860 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-94b9766b-b421-4351-b271-be2d6bb22a25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac5975413f5548c3881ec3c868295836', 'request_id': 'req-94b9766b-b421-4351-b271-be2d6bb22a25'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:49:11.683800 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-94b9766b-b421-4351-b271-be2d6bb22a25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac5975413f5548c3881ec3c868295836', 'request_id': 'req-94b9766b-b421-4351-b271-be2d6bb22a25'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:49:11.691499 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-94b9766b-b421-4351-b271-be2d6bb22a25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac5975413f5548c3881ec3c868295836', 'request_id': 'req-94b9766b-b421-4351-b271-be2d6bb22a25'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:49:11.703626 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-94b9766b-b421-4351-b271-be2d6bb22a25 admin admin] The token cache is being invalidate because project ac5975413f5548c3881ec3c868295836 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:49:11.703919 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-94b9766b-b421-4351-b271-be2d6bb22a25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:49:11.704590 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-94b9766b-b421-4351-b271-be2d6bb22a25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:49:11.706133 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1444/2904] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:49:11 2022] DELETE /identity/v3/projects/ac5975413f5548c3881ec3c868295836 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:49:11.710566 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d50ef0-05d8-4dba-8f15-dd2842e4d1e2 admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:49:11.712021 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-e8d50ef0-05d8-4dba-8f15-dd2842e4d1e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:49:11.726341 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d50ef0-05d8-4dba-8f15-dd2842e4d1e2 admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:49:11.786048 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f566f4-1c5a-4d68-a2a6-61cb03b597ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:49:11.786924 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f566f4-1c5a-4d68-a2a6-61cb03b597ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:49:11.787211 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f566f4-1c5a-4d68-a2a6-61cb03b597ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:49:11.787483 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f566f4-1c5a-4d68-a2a6-61cb03b597ba admin admin] PATH_INFO: `/v3/users/3ad76b4b3d914432bb0a4cf2d9fdcd72` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:49:11.788696 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f566f4-1c5a-4d68-a2a6-61cb03b597ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ad76b4b3d914432bb0a4cf2d9fdcd72)` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:49:11.790597 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f566f4-1c5a-4d68-a2a6-61cb03b597ba admin admin] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:49:11.844337 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a8f566f4-1c5a-4d68-a2a6-61cb03b597ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ad76b4b3d914432bb0a4cf2d9fdcd72'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:49:11.856640 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a8f566f4-1c5a-4d68-a2a6-61cb03b597ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ad76b4b3d914432bb0a4cf2d9fdcd72'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:49:11.866094 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a8f566f4-1c5a-4d68-a2a6-61cb03b597ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ad76b4b3d914432bb0a4cf2d9fdcd72'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:49:11.866641 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a8f566f4-1c5a-4d68-a2a6-61cb03b597ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ad76b4b3d914432bb0a4cf2d9fdcd72'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:49:11.867287 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.notifications [None req-a8f566f4-1c5a-4d68-a2a6-61cb03b597ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ad76b4b3d914432bb0a4cf2d9fdcd72'} {{(pid=80215) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:49:11.870486 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:49:11.884910 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1461/2905] 10.222.0.16 () {58 vars in 1223 bytes} [Mon May 23 12:49:11 2022] DELETE /identity/v3/users/3ad76b4b3d914432bb0a4cf2d9fdcd72 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:49:12.067981 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9766b-b421-4351-b271-be2d6bb22a25 admin admin] Authenticating user token {{(pid=80216) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:49:12.069700 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.fernet_utils [None req-94b9766b-b421-4351-b271-be2d6bb22a25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80216) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:49:12.085437 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9766b-b421-4351-b271-be2d6bb22a25 admin admin] Validating token access rules against request {{(pid=80216) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:49:12.110286 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:49:12.152053 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f0c56-218d-40ae-8a1b-111818f7ed0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89b57772ff384ce6af49426c38787139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c631b5a3911e4647a05ad6138d2bc618', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80216) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:49:12.152884 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f0c56-218d-40ae-8a1b-111818f7ed0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:49:12.153167 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f0c56-218d-40ae-8a1b-111818f7ed0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:49:12.153442 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f0c56-218d-40ae-8a1b-111818f7ed0d admin admin] PATH_INFO: `/v3/projects/2fbc1181e1654578946cfd81c1fdbb3c` {{(pid=80216) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:49:12.154855 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f0c56-218d-40ae-8a1b-111818f7ed0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fbc1181e1654578946cfd81c1fdbb3c)` {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:49:12.157218 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f0c56-218d-40ae-8a1b-111818f7ed0d admin admin] RBAC: Authorization granted {{(pid=80216) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:49:12.207440 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bd4f0c56-218d-40ae-8a1b-111818f7ed0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fbc1181e1654578946cfd81c1fdbb3c', 'request_id': 'req-bd4f0c56-218d-40ae-8a1b-111818f7ed0d'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:49:12.215448 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bd4f0c56-218d-40ae-8a1b-111818f7ed0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fbc1181e1654578946cfd81c1fdbb3c', 'request_id': 'req-bd4f0c56-218d-40ae-8a1b-111818f7ed0d'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:49:12.222266 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bd4f0c56-218d-40ae-8a1b-111818f7ed0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fbc1181e1654578946cfd81c1fdbb3c', 'request_id': 'req-bd4f0c56-218d-40ae-8a1b-111818f7ed0d'} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:49:12.233846 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bd4f0c56-218d-40ae-8a1b-111818f7ed0d admin admin] The token cache is being invalidate because project 2fbc1181e1654578946cfd81c1fdbb3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80216) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 12:49:12.234134 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bd4f0c56-218d-40ae-8a1b-111818f7ed0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:49:12.234786 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG keystone.notifications [None req-bd4f0c56-218d-40ae-8a1b-111818f7ed0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80216) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 12:49:12.236352 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: [pid: 80216|app: 0|req: 1445/2906] 10.222.0.16 () {58 vars in 1232 bytes} [Mon May 23 12:49:12 2022] DELETE /identity/v3/projects/2fbc1181e1654578946cfd81c1fdbb3c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 12:49:21.883405 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=4 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:49:22.230699 dv-cinder-836996-27-iscsi devstack@keystone.service[80216]: DEBUG dbcounter [-] [80216] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80216) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 12:52:17.481946 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f566f4-1c5a-4d68-a2a6-61cb03b597ba admin admin] Authenticating user token {{(pid=80215) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 12:52:17.483413 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-a8f566f4-1c5a-4d68-a2a6-61cb03b597ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:52:17.559177 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f566f4-1c5a-4d68-a2a6-61cb03b597ba admin admin] Validating token access rules against request {{(pid=80215) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 12:52:17.621207 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01aacd-8907-483b-9de4-ea554eabd96c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ebc0f137e0d24dd2a460689bc17a1b98', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '839585501f414e1ab02501422510e54f', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80215) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 12:52:17.621901 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01aacd-8907-483b-9de4-ea554eabd96c service placement] REQUEST_METHOD: `GET` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 12:52:17.622229 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01aacd-8907-483b-9de4-ea554eabd96c service placement] SCRIPT_NAME: `/identity` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 12:52:17.622513 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01aacd-8907-483b-9de4-ea554eabd96c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80215) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 12:52:17.624434 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.fernet_utils [None req-8f01aacd-8907-483b-9de4-ea554eabd96c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80215) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 12:52:17.635787 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f01aacd-8907-483b-9de4-ea554eabd96c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 12:52:17.637446 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f01aacd-8907-483b-9de4-ea554eabd96c service placement] RBAC: Authorization granted {{(pid=80215) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 12:52:17.648059 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: [pid: 80215|app: 0|req: 1462/2907] 10.222.0.16 () {60 vars in 1365 bytes} [Mon May 23 12:52:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 12:52:27.643299 dv-cinder-836996-27-iscsi devstack@keystone.service[80215]: DEBUG dbcounter [-] [80215] Writing DB stats keystone:SELECT=58 {{(pid=80215) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}